Home on FarisZR

How I Made My Homelab Fix Itself Using Komodo and OpenClaw

Wed, 01 Apr 2026 00:00:00 +0000

The most annoying thing about homelabbing is the upkeep. Keeping things running, handling breaking updates, fixing stuff as it breaks instead of letting everything slowly decay. It’s kinda embarrassing telling people how cool your Homelab is, only for things to break constantly.

Luckily AI agents have gotten good enough that you can use them for the upkeep of your homelab.

Until now though, you still had to sit down at your computer, start an agent, copy the logs, show it the error, and unless it’s running on the server itself, apply the fix manually.

You could install the agent directly on the server and let it manage things, but what if you have multiple servers? Like a whole homelab network?

Do you install an agent on every server? And what happens when services depend on each other?

Komodo

Here comes Komodo. Komodo is an open-source GitOps platform that lets you manage your Docker Compose stacks through a clean UI with full API access to logs, container status, and even a bash shell on the server.

It’s a central hub for all your servers. What makes it special is that any change you make to your stacks, resources, or Procedures in Komodo can be committed back to git, so things never drift from the repo. Everything around your compose stacks is described as code under the Resources folder: servers, procedures. That means your agent can also adjust how Komodo itself works on the fly.

My secret sauce: The Komodo KM CLI, agentic edition.

Komodo has a powerful API, but the official CLI doesn’t expose all of it. So I built my own version, the “agentic” edition. It adds support for:

stack - manage stacks and see their status
procedure - trigger procedures like auto-updates
sync - manage resource sync (GitOps repos)
variable - manage variables and secrets without exposing values to the agent (e.g. setting secrets as command outputs)

It also adjusts the exec and ssh commands so agents can run commands on the host or inside a container non-interactively. (ssh for interactive sessions, exec for non-interactive commands.)

Running the KM CLI within a GitOps repo lets your agent debug running homelab services, fix issues, plan and push updates, or even tackle that migration we’ve all been putting off for months.

You can also create a service user so you have an idea what the agent did on komodo.

And if you don’t want your agent running arbitrary commands on your servers, you can disable that at the container or server level in the Komodo “periphery” config. Just install periphery and edit the config to disable terminal access for the host or all containers.

AI Assistants (OpenClaw or Hermes Agent)

Komodo solves the stack access and management problem, but you still have to sit down, open your computer, and prompt the agent to fix things.

What if you could just message your agent and say “Jellyfin is down, fix it”? That’s the idea behind async personal assistants like OpenClaw and the newer Hermes. Fix things as you notice them, wherever you are.

Just clone the git repo, install the agentic KM CLI skill, and your agent can debug and fix any issue across your homelab, even a multi-server network.

Homelabbing becomes genuinely fun. You say what you want and it appears. You can see what changed, revert it easily thanks to git, and even set up backups to guard against catastrophic failures, though most models are smart enough to not do the classic rm -rf mistake anymore. Still nice to have.

Openclaw in action

So how does this look in reality? What can I use this for? Well, I quickly found a use for this.

I self-host my own language tool server and it breaks often. So why not let OpenClaw figure out a solution for it?

After telling it to investigate and actually solve the issue, it deployed a new modified docker stack that added a memory limit to the container

I legit did this while i was cooking in the kitchen, didn’t need to do anything manually.

I also used it to see if any of my servers had LiteLLM installed after the CVE was made public.

The next step: full automation.

Deploying new services and fixing containers on demand is great. But what if things never broke in the first place? What if your agent fixed issues as they happened, not as you noticed them? AKA what if things just Self healed?

it’s actually doable now. All you need is Komodo’s alert system calling a webhook connected to your OpenClaw or Hermes agent. The agent sees the logs and makes whatever changes are needed to bring the service back up. If it’s a server-level issue, you could even hook into your hoster’s API and trigger a force restart. It doesn’t even have to be fully autonomous. You could build a way for the agent to reach you and ask you for approval.

It’s proactive. The agent already has the fix and you are the one to allow it or not. You are actually the manager now.

Honestly, thinking about it feels a bit wild. We’re on our way to self-healing, self-aware systems that can change themselves to stay up no matter what. Give your agent enough access and resources, and it could restore a backup on another server and bring the service back up without you ever noticing. This is an exciting time for sure.

FYI: 2025 Huawei Matepads can't play 4k youtube videos

Mon, 05 Jan 2026 00:00:00 +0000

I got my hands on a 2025 Huawei Matepad pro 12.2, with a gorgeous display. I was kinda stocked to use it as my media power house, that’s where the weirdness started. The device can’t play 4k videos on YouTube for some reason, and to make it worse, it’s a known issue, but it isn’t something that gets mentioned in reviews. It can play 1080p vp9 videos just fine, and you can play 4k video on YouTube through the browser, so what’s going on here?

The reason

Huawei in their ultimate wisdom chose to support the paid h264 and HEVC codecs, but cheeped out on supporting the royalty-free VP9 codec used by the world’s most popular streaming Platform, YouTube. That’s why streaming apps like Netflix can play 4k just fine, because they use HEVC.

YouTube uses VP9, and the Kirin chipset has no hardware VP9 decoder whatsoever. it can only play 1080p vp9 videos because the software decoder is limited to 2048x2048, aka 1920x1080.

Browsers can include their own software decoder, and that’s what happening here, when playing a 4k video, the browser is using the CPU to do the decoding, and that’s why it drops frames on 4k 60fps videos.

What an absolutely asinine decision, what makes it crazier is that VP9 is supported by devices going back to 2013!

Confirmed by Huawei support

I contacted the official Huawei support, and they confirmed that this is a known issue with the workaround being playing videos through the browser.

They also said this issue is known for the 2025 13.3 and 12.2 Matepad Pros and the 2025 Matepad 12X, aka all of their 2025 releases. Great!

A flagship tablet that can’t play 4k videos in 2025, what a flagship!

How I found out

I let OpenCode loose with ADB shell access to the device and “we” systematically checked:

Huawei Browser (4K VP9): dumpsys media.resource_manager showed only software VP9 decoders available (c2.android.vp9.decoder, OMX.google.vp9.decoder). Hardware decoders exist only for H.264 and HEVC. CPU monitoring revealed vpx tile worker threads maxing out cores - clear software decoding.
Codec inventory: Full decoder list from dumpsys media.player confirmed hardware support limited to OMX.hisi.video.decoder.avc (H.264) and OMX.hisi.video.decoder.hevc (HEVC, 4K capable). VP9 is software-only.
Grayjay (1080p AVC): Confirmed hardware acceleration with OMX.hisi.video.decoder.avc allocation and efficient MediaCodec threads.
Grayjay (1080p VP9): Switched to VP9 stream and confirmed degradation to c2.android.vp9.decoder software path. Even at 1080p, no hardware acceleration exists for VP9.

Knocker: Skip the VPN for your homelab, Just Knock

Sun, 14 Dec 2025 00:00:00 +0000

I have a homelab, but accessing it from outside my home network was always a pain. I didn’t want to deal with the hassle of setting up VPNs on every device just to watch something on Jellyfin, so I built Knocker!

It’s a knock-based access control service for your homelab that actually works with mobile apps, with clients on most platforms used by homelabbers.

Your browser doesn't support HTML5 video. Here is a link to the video instead.

How does it work?

By being completely transparent for whitelisted IPs, Knocker doesn’t break API clients like mobile apps. Once you whitelist your IP, you are ready to go. You can even whitelist an IP other than your own if needed.

Knocker Tokens

Knocker works by using tokens as passwords. Each token has specific permissions, like a customizable max TTL (capped server-side) and an option to allow whitelisting IPs other than your own.

By connecting Knocker to your reverse proxy, Knocker will analyze the request IP. Depending on the local whitelists, it will either return a 200 OK or a 401 Unauthorized status, which is then forwarded to the end client.

Why Not Just Use a VPN?

I already use Tailscale. In fact, the IP of my homelab is the Tailscale IP because I added custom routes for it.

But using a VPN for everything is annoying. You have to install it on every single device, which is a nightmare on things like Smart TVs. Plus, keeping the Tailscale app running on Android drains my battery like crazy.

With Knocker, you just need one device to knock, and thanks to NAT, your whole network gets access.

Is This as Secure as a VPN?

No.

Knocker is a compromise. It trades some security for convenience. You can’t whitelist specific devices, only source IPs (which might be shared CGNAT IPs).

You’re basically making a bet that within that short whitelist window, the likelihood of a hacker scanning your specific port and attacking it is pretty slim.

That’s why you should use short TTLs on public networks. In general, Knocker should be used in front of services that already have their own authentication.

Setup

Knocker is distributed as a Docker container that optionally uses the system DBus socket to interact with the FirewallD daemon.

docker-compose.yml: github.com/FarisZR/knocker

knocker.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42


server:
 host: "0.0.0.0"
 port: 8000
 trusted_proxies:
 # Trust the IPv4 and IPv6 subnets of the caddy_net docker network
 # Adjust these to match your specific Docker network configuration.
 - "172.16.238.0/24"
 - "fd00:dead:beef::/64"

cors:
 allowed_origin: "https://knocker.fariszr.com" # SECURITY: Set to your Knocker webapp's origin if you are hosting your own.

security:
 always_allowed_ips:
 # These IPs/CIDRs are always allowed to pass through the /verify endpoint
 # without needing to be dynamically whitelisted.
 # It's recommended to include your reverse proxy's IP here.
 - "172.16.238.0/24"
 - "fd00:dead:beef::/64"
 # Also your home's local network
 - "192.168.1.0/24"
 excluded_paths:
 # Request paths that start with any of these values will bypass
 # the IP whitelist check entirely.
 # Example:
 # - "/api/v1/status"
 # - "/metrics"
 - "/knock"
 # Maximum number of entries in the whitelist (default: 10000)
 max_whitelist_entries: 10000

whitelist:
 # The path where the whitelist file will be stored.
 # This path is relative to the container's file system.
 # The docker-compose.yml file will mount a volume to this location.
 storage_path: "/data/whitelist.json"

api_keys:
 - name: "admin-key-for-remote-whitelisting"
 key: "CHANGE_ME_SUPER_SECRET_ADMIN_KEY" # SECURITY: Change this to a strong, random key
 max_ttl: 3600 # Maximum TTL in seconds (1 hour)
 allow_remote_whitelist: true # Can whitelist other IP/CIDR specified in request body

Connecting Your Reverse Proxy to Knocker

I use Caddy, but Knocker should work with any reverse proxy that supports using an external auth endpoint.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


# Define a reusable snippet for the knock-knock check.
# It points to the knocker service using Docker's internal DNS.
(knocker_auth) {
 forward_auth knocker:8000 {
 uri /verify?
 copy_headers X-Forwarded-For
 }
}

# The public endpoint for performing the knock.
# Make sure this domain points to your Caddy server's IP.
knock.your-domain.com {
 reverse_proxy knocker:8000
}

# An example protected service.
jellyfin.your-domain.com {
 import knocker_auth # Apply the forward_auth check
 reverse_proxy jellyfin_service_name:8096
}

Now your service will respond with a 401 Unauthorized for every non-whitelisted IP.

FirewallD Integration

Knocker has another trick up its sleeve: it can integrate with FirewallD to operate on the firewall level, allowing you to use it for non-HTTP services like a game server!

Your browser doesn't support HTML5 video. Here is a link to the video instead.

There is a caveat, though: exposed bridge Docker ports will still bypass these rules. Unfortunately, there isn’t any firewall that deals with Docker port forwarding rules well. You have to use host networking mode for it to work.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24


firewalld:
 enabled: true
 zone_name: "knocker"
 # WARNING: Zone won't be cleaned up automatically on shutdown
 zone_priority: -100 # Zone priority (negative numbers = higher priority)
 # zone_target: "default" # Optional: Set the zone target (default: not set)
 # Options: "default", "ACCEPT", "REJECT", "DROP"
 # "default" = practically matches reject behavior
 # "ACCEPT" = Accept all packets not matched by rules set by knocker
 # "REJECT" = Reject all packets not matched by rules set by knocker
 # "DROP" = Silently drop all packets not matched by rules set by knocker
 default_action: "drop" # Action for blocked traffic on monitored ports:
 monitored_ports:
 # Only these ports will be protected by knocker firewall rules
 - port: 80
 protocol: tcp
 - port: 443
 protocol: tcp
 - port: 22
 protocol: tcp
 monitored_ips:
 # WARNING: Using 0.0.0.0/0 or ::/0 applies DROP/REJECT rules to ALL traffic
 - "203.0.113.0/24" # Example WAN IPv4 range - CHANGE THIS
 - "2001:db8:1234::/48" # Example WAN IPv6 range - CHANGE THIS

Clients

The strong point of Knocker, and where I actually spent most of my time, is the clients. I designed them to get out of the way as much as possible.

Knocker-Web

GitHub.com/FarisZR/Knocker-Web

Knocker Web is a web client built with Vite. It’s fully static and can be installed as a PWA. It’s meant to be the primary client, as it works basically everywhere.

What makes it really convenient is the “knock on reload” feature. You just open the site and that’s it; you don’t care what happens after that. It’s especially useful when used as a PWA, as it means it will perform a knock when opened.

Knocker-CLI

GitHub.com/FarisZR/knocker-CLI

A CLI client written in Go with support for creating a background service for periodic knocks using Systemd/LaunchAgent.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28


>> knocker --help
A reliable, cross-platform service that keeps your external IP address whitelisted.
It runs in the background, detects IP changes, and ensures you always have access.

Usage:
 knocker [flags]
 knocker [command]

Available Commands:
 completion Generate the autocompletion script for the specified shell
 help Help about any command
 install Install the Knocker service
 knock Manually trigger a whitelist request
 run Run the Knocker service
 start Start the Knocker service
 status Get the status of the Knocker service
 stop Stop the Knocker service
 uninstall Uninstall the Knocker service

Flags:
 --check_interval int Interval in minutes to poll for IP changes (only used when ip_check_url is set) (default 5)
 --config string config file (default is $HOME/.knocker.yaml)
 -h, --help help for knocker
 --ip_check_url string URL of the external IP checker service
 --ttl int Time to live in seconds for the knock request (0 for server default)
 -v, --version version for knocker

Use "knocker [command] --help" for more information about a command.

Knocker-Gnome

GitHub.com/FarisZR/knocker-Gnome

An experimental Gnome extension that interprets the JSON logs output by Knocker-CLI, and allows you to manually trigger knocks, disable the service, and see the timeout for the current knock.

Knocker-EXPO

GitHub.com/FarisZR/knocker-expo

Knocker EXPO

An experimental Android App built using React EXPO. It does the same thing as the PWA, with support for creating a background service to regularly send knocks.

However, its reliability depends on the manufacturer, as not all of them actually allow Android background processes to run reliably.

Vibe Coded with AI

Thanks to the Roo Code hackathon (sponsored by Requestly and Google), I finally had the excuse to build this. I started with Gemini 2.5 Pro, burned through over $1700 in tokens, and eventually moved to GPT-5-CODEX and GitHub Coding Agent (Sonnet 4, later Sonnet 4.5) to build out features.

I used CodeRabbit for reviews and set up a full integration test environment for the agents to iterate against that’s why this thing actually works at all.

I ran static analysis on the backend and found zero vulnerabilities, so there’s that.

What I’m trying to say is, this isn’t your average “I told Replit to code it” project, but still, if you’re anti-AI, don’t use this please.

I may go into more details about my workflow for implementing Knocker using Roo Code in a separate blog post, because it wasn’t easy to go this far with AI.

How to use the Proxy Protocol with Caddy (Get the Source IP from a TCP proxy)

Thu, 16 Oct 2025 00:00:00 +0000

My Homelab often has peering issues when connecting from another ISP. The solution to this is just to pass the connection through a proxy, which has better speed than a direct connection. But if I want to do this without MITMing the connection, it can’t be a classic reverse proxy sandwich.

That’s where tools like rathole come in, they are TCP proxies that allow you to forward any TCP connection to your homelab. However, there’s always the issue of the source IP. Due to how these proxies work, the source will always be the IP of the rathole client on the homelab (AKA the proxy itself) and not the actual source IP of the connection.

Proxy protocol fixes this by appending headers to the TCP proxied connections that give details about the actual source IP.

Using the Proxy Protocol with Caddy

Assuming your TCP proxy supports the proxy protocol (example, modified version of rathole)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


{
 servers {
 listener_wrappers {
 proxy_protocol {
 allow 172.20.0.0/16 #ip with netmask of the proxy
 fallback_policy reject
 }
 tls # the proxied packets are HTTPS, so we still need the tls wrapper https://caddyserver.com/docs/caddyfile/options#proxy-protocol
 }
 }
}

example.fariszr.com {
 reverse_proxy service:8000 {
 header_up X-Forwarded-For {client_ip} # {client_ip} is the source ip in the proxy
 header_up X-Real-IP {client_ip}
 header_up X-Forwarded-Proto {scheme}
 header_up X-Forwarded-Host {host}
 }
}

Now your app should get the source IP correctly without actually being aware of the proxy protocol and the connection being proxied through a TCP proxy.

Gitea Actions, a buggy mess

Wed, 03 Sep 2025 00:00:00 +0000

My team uses Jenkins for CI/CD, and we want to migrate away from it due to its insanely complex and hard to understand scripts.

We use Gitea as our git hosting platform, the direct CI/CD offer for it is Gitea Actions, it’s already there, integrated into Gitea and gives us access to the huge community Ecosystem around GitHub Actions.

Gitea says everything works apart from a few limitations, then everything else should work, right?

https://docs.gitea.com/usage/actions/comparison

WRONG!!!!

Private reusable workflows

In GitHub Actions you can use reusable workflows from private repos, as long as the GitHub token has access to the repo.

Gitea Actions don’t have such a token, but you can just add a PAT to the actions URL for this to work, right?

Wrong, it just doesn’t work no matter what, as the YAML parser doesn’t allow anything other than a specific syntax.

This also affects limited Orgs (Organizations that are only visible to authenticated users). You would think the runner is authenticated, but apparently not.

https://github.com/go-gitea/gitea/issues/25929

Reported in 2023, still broken.

Sure no biggie, we can just create a dedicated public organization for the workflows.

Reusable workflow steps are all shown as one step

Gitea parses the steps through the workflow file, and then sorts the logs under each step. When you are using a reusable workflow, Gitea only parses one step, and all logs are shown under it.

This makes debugging a nightmare when workflows fail.

https://github.com/go-gitea/gitea/issues/26187 Reported in 2023, still broken.

Well, we just have to get used to the mangled logs, maybe just add an emoji to each step, and then search for it in the browser to find the step logs.

Actions are only pulled once

When using a major tag for an action like v4, the latest release belonging to that version will be pulled, including patches and minor releases.

Guess what? That doesn’t work in the Gitea Actions act_runner, since the act_runner has a runtime cache, and the runner won’t check for updates after the first pull.

This is an even bigger problem for reusable workflows, where you might prefer to use a branch instead of tags, and just assume the runner will use the latest commit on each run.

https://gitea.com/gitea/act_runner/issues/726

Simple fix, just tag the actions to the last patch release, and run Renovate to automatically update the tag used, that surely gotta work?

Only for actions hosted on GitHub, not on Gitea.

Renovate can’t update workflow tags in Gitea

https://github.com/renovatebot/renovate/discussions/27734

Renovate still looks for packages in GitHub and not Gitea for reusable workflows and actions.

Meaning auto updates for Gitea-hosted actions/workflows aren’t possible.

Yeah, I’ve had enough, Gitea ain’t it then.

Conclusion

Gitea Actions might work for simple workflows that are only used in one repo, don’t need concurrency limits, don’t need to get updated regularly, and don’t rely on anything non-public.

In addition to these crazy bugs, dealing with HTTP proxies in Gitea Actions is just insane. The NOPROXY proxy exception env variable is very finicky where it’s just ignored most of the time, but I think it’s more of an Actions problem than a Gitea Actions problem.

After hitting so many bugs that were reported years ago! I just can’t continue to invest time in Gitea Actions. Who knows what issues will show up when these workflows become part of the team’s development process? It seems like GitLab is really the way to go.

Vibecoding with GitHub Copilot: Lessons Learned

Thu, 31 Jul 2025 00:00:00 +0000

Over the past few weeks, I’ve been vibecoding a new app with GitHub Copilot.

I know it’s not meant for that, but here’s what I learned anyway:

200 requests are actually a lot

The GitHub Copilot pro sub includes 200 premium requests and unlimited gpt4.1 and gpt-4o requests.

It’s genuinely a lot, especially considering o4-mini’s 0.33x multiplier. It’s great for debugging but not ideal for designing.

Don’t forget that the Copilot coding agent on GitHub uses only one premium request for the entire session.

Not enough with third party extensions

Cline and its forks added support for using copilot as a provider through the VS Code LM API.

However, the way these requests are handled differs significantly. A request in the official plugin is an interaction, every action done by LLM till it stops generating is included in that request.

However, when third party extensions do this each API request counts as a premium request on its own, meaning instead of using one premium request to do the entire generation workflow, it’s using 5 to 10.

That’s at least the case with Cline based tools.

Limited context window

All Copilot models are capped at a 64k context window, which is disappointing, especially for Gemini models that shine with their 1m context window.

(maybe that’s not the case for the async coding agent?)

The Continue to iterate prompt is a disaster

This is the worst thing in GitHub Copilot, it shows up when it’s actually doing work and stops the agent.

This wouldn’t be a problem if the LLM actually just continued iterating, but due to the limited context window it seems to get only a summary of the chat and just starts anew.

It’s very annoying, but luckily there’s a fix, you can increase the request limit, so that it practically never shows up

Context7 MCP is OP

Context7 is an MCP that enables the LLM to fetch up-to-date documentation in Markdown for almost any library out there. Adding it is crucial as it prevents the model from hallucinating or using outdated functions.

Sonnet 4 especially excelled at this, it always used context7 before doing anything, or while debugging.

Other models like o4-mini needed to be reminded explicitly for them to actually use it, either by text or by adding it to the context window.

Stagewise MCP is superb for editing UI elements

Stagewise is an MCP client that adds an npm dev module that allows you to select elements and forward them to Copilot.

This makes adjusting frontend elements much easier.

Generating copilot.md instructions is actually useful

Copilot can now generate a copilot-instructions.md file for itself.

It might sound odd, but it works by providing the LLM with a general project overview, which you can build on with additional instructions.

One long, detailed prompt is way better than refining things later, especially for the Copilot coding agent

This could be a side effect of the small context window, but there’s a huge difference when actually starting with a fully detailed prompt.

The LLM generates based on its assumptions. The more detailed you are, the better these assumptions align with your expectations.

It’s the reason why I think Kiro.dev will excel in this, because it unpacks these assumptions into separate markdown files, which then you can refine.

if you want to use the GitHub Copilot async coding agent, you can trigger it from the chat which will do a search and then write a really detailed prompt. The Results are way better than usual with this.

In general, it’s been decent, but most of the issues really stem from the limited 64k context window, it makes Gemini models almost useless.

However, Copilot’s access to GitHub has been a strong point.

Automatically sync files to Mega.io with dockerized mega-cmd

Mon, 30 Jun 2025 00:00:00 +0000

I self-host my own private cloud, but sometimes I have to make files available on external clouds like Mega.

In this post I will explain how to create a dockerized mega-cmd with an entrypoint script that will automatically sync files and watch for changes.

dockerfile

To make it easier to build the image, i embedded the entrypoint script inside the dockerfile.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90


FROM debian:12

# Install dependencies and MegaCMD
RUN apt-get update && apt-get install -y \
 wget \
 curl \
 gnupg \
 apt-transport-https \
 && mkdir -p /etc/apt/keyrings \
 && curl -fsSL https://mega.nz/keys/MEGA_signing.key | gpg --dearmor -o /etc/apt/keyrings/mega.nz.gpg \
 && echo "deb [arch=amd64,arm64 signed-by=/etc/apt/keyrings/mega.nz.gpg] https://mega.nz/linux/repo/Debian_12/ ./" | tee /etc/apt/sources.list.d/mega.nz.list > /dev/null \
 && apt-get update \
 && apt-get install -y megacmd \
 && apt-get clean \
 && rm -rf /var/lib/apt/lists/*

# Create directories for configuration and data
RUN mkdir -p /root/.megaCmd /data

# Create startup script
RUN echo '#!/bin/bash\n\
set -e\n\
\n\
# Function to check if already logged in\n\
check_login() {\n\
 mega-whoami > /dev/null 2>&1\n\
 return $?\n\
}\n\
\n\
# Check if already logged in\n\
if check_login; then\n\
 echo "Existing session found. Skipping login."\n\
else\n\
 # Check if credentials are provided\n\
 if [ -z "$MEGA_EMAIL" ] || [ -z "$MEGA_PASSWORD" ]; then\n\
 echo "Error: MEGA_EMAIL and MEGA_PASSWORD environment variables must be set"\n\
 exit 1\n\
 fi\n\
\n\
 # Login to MEGA\n\
 if [ -n "$MEGA_2FA" ]; then\n\
 mega-login $MEGA_EMAIL $MEGA_PASSWORD --auth-code=$MEGA_2FA\n\
 else\n\
 mega-login $MEGA_EMAIL $MEGA_PASSWORD\n\
 fi\n\
fi\n\
\n\
# Show account info to confirm login\n\
echo "Logged in as:"\n\
mega-whoami\n\
\n\
# Set up sync if MEGA_SYNC_LOCAL_PATH and MEGA_SYNC_REMOTE_PATH are provided\n\
if [ ! -z "$MEGA_SYNC_LOCAL_PATH" ] && [ ! -z "$MEGA_SYNC_REMOTE_PATH" ]; then\n\
 # Check if sync already exists\n\
 if mega-sync | grep -q "$MEGA_SYNC_LOCAL_PATH"; then\n\
 echo "Sync already exists for $MEGA_SYNC_LOCAL_PATH. Skipping sync setup."\n\
 else\n\
 echo "Setting up sync from $MEGA_SYNC_LOCAL_PATH to $MEGA_SYNC_REMOTE_PATH"\n\
 mega-sync $MEGA_SYNC_LOCAL_PATH $MEGA_SYNC_REMOTE_PATH\n\
 fi\n\
fi\n\
\n\
# Keep the container running and show logs\n\
echo "MegaCMD is now running and syncing..."\n\
echo "Streaming MegaCMD logs..."\n\
\n\
# Wait a moment for log files to be created\n\
sleep 2\n\
\n\
# Tail the mega-cmd log files to show activity in docker logs\n\
# MegaCMD creates log files in ~/.megaCmd/\n\
if [ -f /root/.megaCmd/megacmdserver.log ]; then\n\
 tail -f /root/.megaCmd/megacmdserver.log &\n\
fi\n\
\n\
if [ -f /root/.megaCmd/megacmd.log ]; then\n\
 tail -f /root/.megaCmd/megacmd.log &\n\
fi\n\
\n\
# Also monitor any sync-related logs\n\
if [ -d /root/.megaCmd/logs ]; then\n\
 tail -f /root/.megaCmd/logs/*.log 2>/dev/null &\n\
fi\n\
\n\
# Keep the main process alive\n\
wait\n\
' > /entrypoint.sh && chmod +x /entrypoint.sh

# Set the entrypoint
ENTRYPOINT ["/entrypoint.sh"]

docker-compose.yml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


services:
 megacmd:
 build: .
 restart: always
 environment:
 - MEGA_EMAIL=your@email.com
 - "MEGA_PASSWORD=pass"
 - MEGA_2FA=10101
 - MEGA_SYNC_LOCAL_PATH=/data # directory to sync inside the container
 - MEGA_SYNC_REMOTE_PATH=/remote-dir # directory on mega
 - MEGA_DEVICE_ID=dockerized-mega #device name
 volumes:
 - ~/files:/data
 - ./megacmd-config:/root/.megaCmd
 - /etc/machine-id:/etc/machine-id:ro

the authentication variables are only needed for the first run, after that the token is stored in the .megaCmd folder.

And now your files should show up on mega!

Keep in mind mega support only two-way sync, mounting the data as read only isn’t an option due to megacmd checking for write-access on startup.

How to add support for Element Call with Docker Compose & Caddy

Tue, 27 May 2025 00:00:00 +0000

If you have been getting MISSING_MATRIX_RTC_FOCUS on Element X recently when starting calls, the reason is that Element has stopped their free relay for Matrix RTC, AKA Element Call. It was there just to speed up initial Element X adoption.

Luckily, adding it isn’t that hard. The tricky part is the reverse proxy and updating the well-known files for the clients, but don’t worry, I already did the hard work for you 😅

We need two domains: one for LiveKit (the RTC service) and one for the JWT auth service for Element X.

Docker Compose additions

(livekit_api_key) = api key name

(livekit_api_secret) = random 20+ character string.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41


 # JWT Authentication Service for Element Call
 lk-jwt-service:
 image: ghcr.io/element-hq/lk-jwt-service:latest
 restart: unless-stopped
 environment:
 # Port the JWT service listens on internally
 - LK_JWT_PORT=8080
 # Publicly accessible URL for the LiveKit SFU WebSocket endpoint (updated domain)
 - LIVEKIT_URL=wss://matrixrtc.fariszr.com
 # API Key and Secret (must match LiveKit service and secrets)
 - LIVEKIT_KEY=(livekit_api_key)
 - LIVEKIT_SECRET=(livekit_api_secret)
 # Optional: Restrict call creation to users from specific homeservers
 - LIVEKIT_LOCAL_HOMESERVERS=fariszr.com # server name
 networks:
 default:
 web: # Needs to be reachable by Caddy

 livekit:
 image: livekit/livekit-server:v1.8
 container_name: livekit
 # Use config file instead of just environment variables
 command: --config /etc/livekit.yaml
 restart: unless-stopped
 # LIVEKIT_KEYS is still needed here for the server itself
 environment:
 LIVEKIT_KEYS: "(livekit_api_key): (livekit_api_secret)"
 # Remove other env vars now handled by config.yaml
 # - LIVEKIT_WS_URL=wss://livekit.fariszr.com
 # - LIVEKIT_PORT=7880
 ports:
 # Port 7880 is now only needed internally for Caddy proxying to /livekit/sfu
 # - "7880:7880" # Client API and Webhook callback - proxied via Caddy
 - "7881:7881" # TURN/TLS (LiveKit's own TURN, though disabled in config)
 - "50100-50200:50100-50200/udp" # UDP port range for media over WebRTC
 volumes:
 # Mount the new config file
 - ./livekit-config.yaml:/etc/livekit.yaml:ro
 networks:
 default:
 web: # Needs to be reachable by Caddy

Configs

Additions to the Synapse config:

homeserver.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


experimental_features:
 # MSC3266: Room summary API. Used for knocking over federation.
 msc3266_enabled: true
 # MSC4222: Needed for syncv2 state_after. Allows clients to correctly track room state.
 msc4222_enabled: true
 # MSC4140: Delayed events are required for proper call participation signalling.
 # If disabled, you might get stuck calls in Matrix rooms.
 msc4140_enabled: true

# The maximum allowed duration by which sent events can be delayed, as per MSC4140.
max_event_delay_duration: 24h

# Adjust rate limiting for call-related events
rc_message:
 # Needs to match at least E2EE key sharing frequency plus headroom.
 # Key sharing events are bursty.
 per_second: 0.5
 burst_count: 30

rc_delayed_event_mgmt:
 # Needs to match at least the heartbeat frequency (5s = 0.2/s) plus headroom.
 per_second: 1.0 # Increased from example's 1 to be safer
 burst_count: 20

livekit-config.yaml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27


# Basic LiveKit configuration
port: 7880 # Port LiveKit listens on internally
bind_addresses:
 - "0.0.0.0" # Listen on all interfaces within the container

rtc:
 tcp_port: 7881
 port_range_start: 50100 # Match docker-compose port range
 port_range_end: 50200 # Match docker-compose port range
 use_external_ip: false # Rely on reverse proxy/NAT

logging:
 level: info

# Disable LiveKit's internal TURN server, since you probably have something already set up for the classic P2P calls.
turn:
 enabled: false
 # domain: matrixrtc.fariszr.com # Not needed if disabled, updated comment for consistency
 # cert_file: ""
 # key_file: ""
 # tls_port: 5349 # Default
 # udp_port: 443 # Default
 # external_tls: true # If using external certs, but disabled anyway

# Keys are provided via environment variable LIVEKIT_KEYS in docker-compose.yml
# keys:
# (livekit_api_key): (livekit_api_secret)

Caddy reverse proxy

I had to spend a lot of time figuring out the best reverse proxy setup. It seems that proxying the JWT service and LiveKit on different domains is the easiest solution, because I couldn’t get it to work with a subdirectory override for JWT on the same domain.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


matrix.fariszr.com {
 reverse_proxy synapse:8008
 encode zstd gzip
 # Serves the Matrix client .well-known JSON configuration file
 respond /.well-known/matrix/client `{"m.homeserver":{"base_url":"https://matrix.fariszr.com"},"org.matrix.msc4143.rtc_foci":[{"type":"livekit","livekit_service_url":"https://jwt.matrixrtc.fariszr.com"}]`
}

jwt.matrixrtc.fariszr.com {
 encode zstd gzip
 reverse_proxy lk-jwt-service:8080
}

matrixrtc.fariszr.com {
 encode zstd gzip
 # Route SFU WebSocket requests to the livekit container on its internal port 7880
 reverse_proxy livekit:7880
}

IMPORTANT: If you are using your delegated domain on the client (i.e., fariszr.com instead of matrix.fariszr.com when logging in), then you will have to update the client well-known file served on the delegated domain too.

And now you should have a Matrix RTC-capable home server!

You can take a look at my server’s setup anytime on my GitOps repo, run with docker-compose-gitops-action.

Sources

https://github.com/element-hq/element-call/blob/livekit/docs/self-hosting.md

https://willlewis.co.uk/blog/posts/deploy-element-call-backend-with-synapse-and-docker-compose

Selfhosted grammar autocorrect with languagetool and docker compose.

Mon, 31 Mar 2025 00:00:00 +0000

Self-hosted grammar autocorrect with LanguageTool and Docker Compose.

LanguageTool is a great extension for correcting typos and grammar errors, it’s basically a more advanced and useful version of autocorrect.

However, LanguageTool watches everything you type in real time, so it’s a real privacy risk.

Fortunately, LanguageTool is open source, at least the core of it, and you can easily host it yourself using Docker.

Setup

Download the N-gram language weights

Ngrams are large datasets that help Languagetool improve context-dependent corrections, such as yours and there.

Languagetool offers free n-gram datasets for a few languages, but AFAIK they are not the same as those used in the cloud version, and certainly not in the premium version.

https://languagetool.org/download/ngram-data/

Download and extract the N-gram datasets into a language-specific folder under the N-grams directory, etc. ./ngrams/en, ./ngrams/de.

Docker-compose.yml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21


services:
 languagetool:
 image: erikvl87/languagetool:latest
 tmpfs:
 - /tmp:exec
 cap_drop:
 - ALL
 cap_add:
 - CAP_SETUID
 - CAP_SETGID
 - CAP_CHOWN
 security_opt:
 - no-new-privileges
 ports:
 - 100.81.157.127:8010:8010
 environment:
 - langtool_languageModel=/ngrams # OPTIONAL: Using ngrams data
 - Java_Xms=512m # OPTIONAL: Setting a minimal Java heap size of 512 mib
 - Java_Xmx=2g # Max memory 
 volumes:
 - ./ngrams:/ngrams

Add Fasttext

Contrary to its name, Fasttext is not about improving correction speed, it’s an algorithm to improve autocorrect accuracy using an open model from Facebook.

Dockerfile

1
2
3
4


FROM erikvl87/languagetool:latest
USER root
RUN apk add fasttext
USER languagetool

download the fasttext Model

fasttext also needs a model to work: https://fasttext.cc/docs/en/language-identification.html

Save it in the same LanguageTool directory and rename it to fasttext-model.bin.

Fully featured Docker-compose

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27


services:
 languagetool:
 # image: erikvl87/languagetool:latest
 build: .
 container_name: languagetool
 restart: always
 tmpfs:
 - /tmp:exec
 cap_drop:
 - ALL
 cap_add:
 - CAP_SETUID
 - CAP_SETGID
 - CAP_CHOWN
 security_opt:
 - no-new-privileges
 ports:
 - 8010:8010
 environment:
 - langtool_languageModel=/ngrams # OPTIONAL: Using ngrams data
 - Java_Xms=512m # OPTIONAL: Setting a minimal Java heap size of 512 mib
 - Java_Xmx=3g
 - langtool_fasttextBinary=/usr/bin/fasttext
 - langtool_fasttextModel=/fasttext-model.bin
 volumes:
 - ./ngrams:/ngrams
 - ./fasttext-model.bin:/fasttext-model.bin

Make the web extensions use your selfhosted instance

Go to languagetool settings, scroll all the way down to pro settings, and then enter the URL of your instance with /v2 appended at the end. If you are hosting it on the same machine, then it is https://127.0.0.1:8010/v2.

BTW you can use languagetool in other applications like Vscode/Vscodium and Libreoffice.

Sources

https://github.com/Erikvl87/docker-languagetool/issues/70 https://github.com/Erikvl87/docker-languagetool

Go Paperless with Paperless-ngx and Ai + Nextcloud Intergration.

Fri, 28 Feb 2025 00:00:00 +0000

Having a scanner is the first step of going paperless, however with time you realize that keeping stuff in order and finding documents quickly becomes a challenge, even when using full text search in the cloud.

Thats why document management Systems like Paperless-ngx exist. And in this post I’m going to setup paperless-ngx with Nextcloud Integration, automated AI tagging and more accurate LLM-powered OCR.

Docker-compose

For the docker compose setup i don’t have to explain much, Paperless-ngx has template docker files ready to use in their Git repos and a quick guide on how to use them: https://docs.paperless-ngx.com/setup/#docker

The needed changes will be mentioned in the next steps.

setting up separate directories for each user

unless you want every user to be able to see each others documents, you need to make sure paperless separates the resulting files into a per-user dir.

You can do this with workflows and storage Paths, or make it the default and just use workflows to assign documents to users when importing(“consuming”) docs.

Make Paperless-ngx use a separate directory for each user.

Using the placeholder details in the Paperless docs, you can set the default file format in Paperless using an environment variable.

1
2
3
4
5
6


services:
 app:
 ....
 environment:
 .....
+ PAPERLESS_FILENAME_FORMAT='{{ owner_username }}/{{ title }}'

This will keep the filename and store the file in an owner-specific directory in the paperless archive folder

Assign ownership based on subdirectory in consumption directory

By default, when importing from the consumption directory, the file is available to all users, but with workflows we can tell Paperless to assign ownership based on the file dir when importing.

If you have a scanner/printer that supports Samba, you can have it scan directly to a user-specified SMB share. A guide to setting up an SMB share with Docker can be found here.

Workflow trigger

Trigger type: Consumption started Filter Sources: Consumption folder

Filter path: /usr/src/paperless/consume/$USERNAME This matches any file imported from a specific directory

you can also use wildcards to match any file with the username in its directory by just writing *$USERNAME*

Workflow action

the action should be to assign the ownership and rights to the user.

AI with Paperless-GPT

It’s 2025, we need to have AI in everything right?

It makes sense here though, with Paperless-GPT you can use it to OCR documents using vision LLM models, using cloud vision models like GPT-4o, custom services from Google and Azure or even using local open source LLMs!

Install paperless-GPT

https://github.com/icereed/paperless-gpt?tab=readme-ov-file#installation

Assuming you want to use ollama, make sure you adjust the LLM_PROVIDER and LLM_MODEL variables accordingly. Also comment out the OPENAI_API_KEY environment variable.

For PAPERLESS_API_TOKEN, you can generate a token by going to the django admin dashboard, which you can find in the settings page under your profile icon in the top right.

(You can also use vLLM, in which case you should probably stick with openai as your provider, while setting the OPENAI_BASE_URL to point to vLLM).

You can also customize the tags used to trigger processing by paperless-gpt, whether just for metadata or for OCR.

Paperless-gpt has a webui exposed on port 8080 by default, consider setting up a reverse proxy in front of it. Example for caddy:

1
2
3


paperless-ai.yourdomain.test {
 reverse_proxy paperless-gpt:8080
}

Ollama setup

Ollama is the easiest way to run LLMs locally, but its support for Vision LLMs is very limited, with the latest model being minicpm-v 2.6, which doesn’t work well with non-Latin languages. If you want to run better models like Qwen2.5-VL-7B-Instruct, you will have to use vLLM for that. (if you figure out how to make it run on consumer AMD GPUs [not the 7900xt/x], hit me up)

ROCm (AMD)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


services:
 ollama:
 image: ollama/ollama:rocm
 devices:
 - /dev/kfd:/dev/kfd
 - /dev/dri:/dev/dri
 # environment:
 # - HSA_OVERRIDE_GFX_VERSION=10.3.0 # for unsupported gpus such as the 6700xt, 7800xt etc.
 volumes:
 - ./ollama:/root/.ollama
 ports:
 - "11434:11434"

CUDA (Nvidia)

(If you have an Nvidia GPU, You should probably be using vLLM for this) You will need to install the nvidia container toolkit and enable it for docker, see the Ollama’s docker hub page for details.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


services:
 container_name: ollama
 ollama:
 deploy:
 resources:
 reservations:
 devices:
 - driver: nvidia
 count: all
 capabilities:
 - gpu
 volumes:
 - ollama:/root/.ollama
 ports:
 - 11434:11434
 image: ollama/ollama

pulling the llm model

1
2


docker exec -it ollama bash
~# ollama pull minicpm-v

Testing out OCR and Tag suggestions using minicpm-v

Open the paperless-gpt dashboard (exposed on port 8080 by default) and check that everything is working correctly.

To test OCR with a Vision LLM, go to the OCR tab and enter the ID of a document you want to test on.

You can find the ID in the URL of each document, https://paperless.yourdomain.test/documents/XXX/details, XXX being the document id.

To test the metadata suggestions (based on the text extracted by teserract), you have to add one of the paperless-gpt processing tags to the document, by default paperless-gpt for manual processing (you have to trigger it in the UI) and paperless-gpt-auto for automatic processing (for OCR its paperless-gpt-ocr-auto by default). You can customize the tag names using environment variables in docker compose.

When documents are tagged with paperless-gpt, they are displayed in the web interface.

You can now create better metadata based on the OCR text (from Paperless-NGX).

Process all documents with Paperless-gpt

If you want all documents to be processed by AI, you can create a workflow to add the paperless-gpt-auto' and paperless-gpt-ocr-auto’ tags to new documents. They will then be automatically processed by paperless-gpt and updated accordingly.

General Tips on using Paperless

I found this thread on reddit to be useful on how to get started with Paperless:

https://reddit.com/r/selfhosted/comments/sdv0rr/paperless_ng_which_tags_document_types/

And now you have an (AI-assisted) Document management System!

Nextcloud Integration

Now if you are setting up paperless, chances are, you are using a cloud solution like nextcloud.

You can integrate Nextcloud with Paperless through external mounts, you can mount the user-specfic consumption directory for each user, and then the archive directory to be able to access processed pdfs.

/mnt/archive being a mounted directory pointing to the {YOUR_DOCKER_DIR}/paperless-ngx/media/documents/archive/$USER$:/mnt/archive:ro and scans pointing to a directory mounted as /usr/src/paperless/consume/$USER in paperless.

Make sure to set per-user paths as the default or to create an active workflow to assign storage paths depending on the import-path.

Sources

https://www.madebyagents.com/blog/paperless-ngx-nextcloud-integration

https://github.com/icereed/paperless-gpt

Setup Contact and Calendar (dav)sync using Radicale and Docker compose.

Sat, 30 Nov 2024 00:00:00 +0000

I have been looking for a solution to sync my contacts across devices. Etesync seemed great for this, but it seems the IOS app is no longer maintained.

And then I found Radicale, a lightweight and easy to setup server for Caldav/Cardav syncing, and in true FarisZR fashion, I’m going to install it with docker compose.

Now radicale doesn’t seem to have an official docker image, but luckily for us tomesquest did the hard part for us and created a docker image for Radiacle.

docker-compose.yml

*Note that this uses an external network called web to connect the service to the reverse proxy, you could just expose the ports directly.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36


networks:
 web:
 external: true

services:
 radicale:
 image: tomsquest/docker-radicale:latest
 container_name: radicale
 # ports:
 # - 127.0.0.1:5232:5232
 init: true
 read_only: true
 security_opt:
 - no-new-privileges:true
 cap_drop:
 - ALL
 cap_add:
 - SETUID
 - SETGID
 - CHOWN
 - KILL
 deploy:
 resources:
 limits:
 memory: 256M
 pids: 50
 healthcheck:
 test: curl -f http://127.0.0.1:5232 || exit 1
 interval: 30s
 retries: 3
 restart: unless-stopped
 volumes:
 - ./data:/data
 - ./config:/config
 networks:
 - web

config

There’s a template config file for the docker image, we still need to change something to be able to set up htpasswd auth.

Create a file named config inside the mounted config directory with the raw content of the config file from github

and modify it to your liking, if you want to change the data directory, make sure to adjust the mount points in docker compose accordingly.

Auth

We need to adjust the auth section to enable authentication with a password, here’s an example:

1
2
3
4


[auth]
type = htpasswd
htpasswd_filename = /config/users
htpasswd_encryption = bcrypt

Hashing your password with bcrypt

Now, as you probably noticed, htpasswd encryption is set to bcrypt, which means we need to hash the password in bcrypt format.

1

echo -n "your password" | mkpasswd --method=bcrypt

Then create a file called users in the mounted config directory. It should be formatted like this:

1

john:$2a$10$l1Se4qIaRlfOnaC1pGt32uNe/Dr61r4JrZQCNnY.kTx2KgJ70GPSm

and now it should be ready to run!

Caddy reverse proxy

1
2
3
4


sync.example.com {
 reverse_proxy radicale:5232
 encode zstd gzip
}

Importing data

For me, importing data from the web UI didn’t work, I just re-imported my contacts on my phone and it worked.

Quick Fixes for Flatpak Dark Mode Detection Issues

Mon, 28 Oct 2024 00:00:00 +0000

Flatpak is the most popular universal distribution format on Linux, but I still face issues with dark mode detection in some apps.

Luckily I found some workarounds to fix this.

Make sure dark mode themes are installed for flatpak updates

Technically, flatpak should automatically detect the theme you’re using and install it, but if for some reason it doesn’t, you can manually install the gtk3 themes you need.

Adwaita:

1

flatpak install org.gtk.Gtk3theme.Adwaita-dark/x86_64/3.22

Generally for Ubuntu: dark

1

flatpak install flathub org.gtk.Gtk3theme.Yaru-dark

light

1

flatpak install flathub org.gtk.Gtk3theme.Yaru-light

But these are the general themes, if you’ve changed the accent colour then you’re using a different theme, like Yaru-Red-dark for the red accent theme.

You can find all the available gtk3themes on Flathub using the command

1

flatpak search gtk3theme

Switch to the stock Adwaita theme

Open the Gnome Tweaks application and make sure you are using the stock Adwaita theme for Gnome. The default Adwaita theme provides the best compatibility, Some applications like Proton VPN seem to only follow dark mode when using Adwaita.

Manual overrides

Some applications don’t implement the necessary XDG standards to detect dark mode, at least not completely. These fixes will manually override the default theme, which means that dynamic switching between dark and light themes from GNOME’s quick settings won’t work anymore when using these overrides.

Use adw-gtk3

Adw-gtk3 is a gtk3 theme that makes applications look like gtk4. It’s also the theme that fixes all the problems for me. Apps like Xournal++ won’t follow the dark mode with Adwaita, even when I manually select adwaita-dark in gnome-tweaks, but with adw-gtk3-dark it works fine.

Even applications like Motrix, which use an old version of electron that doesn’t support automatic dark mode detection, detected dark mode when using adw-gtk3-dark.

Adw-gtk3 Installaion instructions

Manual override with Flatseal

If nothing works, you can manually override the theme used by the applications using the GTK_THEME environment variable.

You can also do this at the application level.

These are the workarounds I found, for me adw-gtk3 fixed everything and I don’t have to worry about this anymore, however, because it is explicitly selected in gnome-tweaks, the light/dark switch in the quick settings will not work..

Sources

https://ubuntuhandbook.org/index.php/2021/10/enable-dark-flatpak-apps-ubuntu-linux-mint/ https://itsfoss.com/flatpak-app-apply-theme/ https://github.com/lassekongo83/adw-gtk3

Using Custom providers like Groq with Zed Ai

Sun, 08 Sep 2024 00:00:00 +0000

Zed is a new open source code editor on the block, which has a top-notch Ai integration, aiming to compete with Cursor.

You can use Zed Ai for free, it will use Claude 3.5 sonnet with strict rate limits, and it also may use your Inputs for training.

However you can also run Zed AI using openAI LLMs or even any other LLM running on an openai api-compatible provider like groq!

settings.json

Zed’s settings are mainly managed through the settings.json file. Open the file using the shortcut ctrl +,

And paste the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


{
 "language_models": {
 "openai": {
 "version": "1",
 "api_url": "https://api.groq.com/openai/v1",
 "available_models": [
 {
 "name": "llama-3.1-70b-versatile",
 "max_tokens": 131072,
 "max_output_tokens": null
 }
 ]
 }
 }
}

api_url: openai compatible API url, for groq it’s https://api.groq.com/openai/v1
available_models: this is a json array of all models from this provider, the best available model on groq is LLama 3.1 70b

storing your API token and enabling choosing custom LLM model

Enter your API token for Groq or your preferred OpenAI-compatible platform in the OpenAI modal at the end of the zed ai configure page.

You can generate your Groqcloud API token by going to the groq console page and then to api keys

then choose your custom LLM model (it’s going to have an OpenAI logo next to it, sorry!) and now you should have a Llama-powered Zed AI assistant!

sources

https://github.com/zed-industries/zed/pull/13276

How to run Brave Leo LLM locally with ollama and Docker compose

Sat, 31 Aug 2024 00:00:00 +0000

Brave recently introduced the ability to bring your own model to use with LEO using other third party providers or a local model using Ollama!

This is a quick guide to hosting Ollama with docker and integrating it with Leo. I’m doing this with docker because it’s much easier for AMD GPUs, because you don’t need any drivers to make it work.

Ollama setup

CPU

1
2
3
4
5
6
7


services:
 ollama:
 image: ollama/ollama
 volumes:
 - ./ollama:/root/.ollama
 ports:
 - "11434:11434"

ROCm

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


services:
 ollama:
 image: ollama/ollama:rocm
 devices:
 - /dev/kfd:/dev/kfd
 - /dev/dri:/dev/dri
 # environment:
 # - HSA_OVERRIDE_GFX_VERSION=10.3.0 # for unsupported gpus such as 6700xt, 7800xt etc.
 volumes:
 - ./ollama:/root/.ollama
 ports:
 - "11434:11434"

CUDA

You will need to install the nvidia container toolkit and enable it for docker, see the docker hub page for Ollama for details.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


services:
 container_name: ollama
 ollama:
 deploy:
 resources:
 reservations:
 devices:
 - driver: nvidia
 count: all
 capabilities:
 - gpu
 volumes:
 - ollama:/root/.ollama
 ports:
 - 11434:11434
 image: ollama/ollama

Downloading an LLM

You need to choose a model that fits into your GPU’s VRAM. Remember that there are smaller “quantized” versions of most LLMs, you could just scroll down the tag picker on Ollama’s site until you see something that might fit your GPU.

You can find the state-of-the-art models using the Lmsys’s chatbot arena Leaderboard, which ranks most Ai models depending on how users like them. https://lmarena.ai/

Meta AI’s LLama 3 models are currently by far the most popular models on Ollama.

1
2


docker exec -it ollama bash
~# ollama pull llama3:YOUR_TAG

This will download the model to the mounted ./ollama directory.

Using Leo with Ollama

Go to settings and then to the Leo page (pictures courtesy of brave)

The model request name is the same as the one in Ollama, so llama3:YOUR_TAG

the server endpoint will always be: http://localhost:11434/v1/chat/completions unless you host Ollama somewhere else.

And then select the model when using leo,

And now you have your own local AI assistant built into your browser!

Setup Owncloud infinite scale with docker compose

Wed, 31 Jul 2024 00:00:00 +0000

ownCloud infinite scale is a complete rewrite of the classic ownCloud/Nextcloud server in go, it’s much faster and much more efficient, to the point it doesn’t even need a database!

I found their docs to be a bit confusing, but I managed to figure it out and wrote this guide to save you the hassle!

Setup plan

ownCloud infinite scale is, as the name says, very scalable, you could deploy multiple instances, with services separated from the main program to be able to scale depending on usage and so on.

This guide is focused on the most common setup for Homelabs and small scale NAS’s, local UNIX storage with one instance using the embedded supervisor. It’s also a great starting point if you are planning a bigger deployment.

Initial ownCloud setup

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41


services:
 ocis:
 image: owncloud/ocis:5
 user: 1000:1000
 ports:
 - 127.0.0.1:9200:9200
 entrypoint:
 - /bin/sh
 # run ocis init to initialize a configuration file with random secrets
 # it will fail on subsequent runs, because the config file already exists
 # therefore we ignore the error and then start the ocis server
 command: ["-c", "ocis init || true; ocis server"]
 environment:
 OCIS_URL: https://owncloud.yourdomain.com
 OCIS_LOG_LEVEL: error # make oCIS less verbose
 PROXY_TLS: false # do not use SSL between reverse proxy and oCIS
 OCIS_INSECURE: true
 # basic auth (not recommended, but needed for eg. WebDav clients that do not support OpenID Connect)
 PROXY_ENABLE_BASIC_AUTH: false
 # admin user password
 IDM_ADMIN_PASSWORD: "admin" # this overrides the admin password from the configuration file
 # make settings service available to oCIS Hello
 SETTINGS_GRPC_ADDR: 0.0.0.0:9191
 GATEWAY_GRPC_ADDR: 0.0.0.0:9142 # make the REVA gateway accessible to the app drivers
 # email server (if configured)
 NOTIFICATIONS_SMTP_HOST: "xxxxxx"
 NOTIFICATIONS_SMTP_PORT: "xxxx"
 NOTIFICATIONS_SMTP_SENDER: "xxxxx"
 NOTIFICATIONS_SMTP_USERNAME: "xxxxxxxx"
 NOTIFICATIONS_SMTP_INSECURE: "xxxxxxx"
 # PROXY_TLS is set to "false", the download url has no https
 STORAGE_USERS_DATA_GATEWAY_URL: http://ocis:9200/data
 # separate directory for thumbnails
 THUMBNAILS_FILESYSTEMSTORAGE_ROOT: /var/lib/ocis-thumbnails
 volumes:
 - ./ocis-config:/etc/ocis
 - ./ocis-data:/var/lib/ocis
 - ./thumbnails:/var/lib/ocis-thumbnails
 logging:
 driver: "local"
 restart: always

This is the basic setup for OCIS running on port 9200, with a separate directory for thumbnails (useful for speeding up thumbnail loading by storing them on an SSD) and override for the admin password.

You need a reverse proxy for this to work properly, you could use Caddy or Nginx for this.

This includes all the basic ownCloud services, you could just continue to use it like this if you feel it’s adequate for your use case.

But you could add a few bells and whistles to your instance by adding a few optional services:

Enable full text search

By default the text search in OCIS only indexes file and folder names and the content of raw and Markdown text files. You can enable Full text search by adding Apache Tika, which will use OCR to read PDF file content and auto-detect its language (list of supported languages) to build a search index of file contents.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


services:
 ocis:
 environment: #add this to your yaml file!
 # better fulltext search
 SEARCH_EXTRACTOR_TYPE: tika
 SEARCH_EXTRACTOR_TIKA_TIKA_URL: http://tika:9998
 FRONTEND_FULL_TEXT_SEARCH_ENABLED: "true"

# better full text search
 tika:
 image: apache/tika:latest-full
 restart: always
 # command: apt update && apt install tesseract-ocr-xxx -y add additional languages (only works if Tika supports auto-detection for that language)
 logging:
 driver: local

This is a partial compose file, you should add the environment variables to the OCIS template at the start of the article and the Tika container.

Automatic anti virus scanning

If you don’t like viruses casually hanging up with your files on your server, you can enable the antivirus service using ClamAV as an antivirus scanner.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26


services:
 ocis:
 enviroment:
 # ClamAV setup
 ANTIVIRUS_SCANNER_TYPE: clamav
 ANTIVIRUS_INFECTED_FILE_HANDLING: delete
 ANTIVIRUS_CLAMAV_SOCKET: "/var/run/clamav/clamd.sock"
 # enable the antivirus service
 OCIS_ADD_RUN_SERVICES: "antivirus"
 # configure the antivirus service
 POSTPROCESSING_STEPS: "virusscan"
 volumes:
 - clamav-socket:/var/run/clamav

 clamav:
 image: clamav/clamav:latest
 container_name: clamav
 # restart: always
 # ports:
 # - '127.0.0.1:3310:3310'
 volumes:
 - ./clamav:/var/lib/clamav
 - clamav-socket:/tmp

volumes:
 clamav-socket:

Online office

One of the latest features in OCIS 6 is the addition of support for online office suites like Collabora (Libreoffice) and OnlyOffice.

However, OCIS 6 is a rolling release, which doesn’t enjoy full production support and is intended for “Early adopters”.

So you will have to switch to the rolling release for this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63


services:
 ocis:
 image: owncloud/ocis-rolling:latest # use OCIS-rolling releases
 enviroment:
 NATS_NATS_HOST: 0.0.0.0 # make NATS accessible to ocis-collaboration
 NATS_NATS_PORT: 9233
 # make collabora the secure view app
 FRONTEND_APP_HANDLER_SECURE_VIEW_APP_ADDR: com.owncloud.api.collaboration.Collabora
 # fix CSP for collabora
 PROXY_CSP_CONFIG_FILE_LOCATION: /etc/ocis/csp.yaml
 volumes:
 - ./csp.yaml:/etc/ocis/csp.yaml # modifiy CSP policy to allow loading of Collabora in OCIS web ui

# the collaboration service isn't included in the embedded supervisor and needs to be run separately
 ocis-collaboration:
 image: owncloud/ocis-rolling:latest
 user: 1000:1000
 restart: always
 ports:
 - 127.0.0.1:9300:9300 #woopi server port
 depends_on:
 ocis:
 condition: service_started
 collabora:
 condition: service_healthy
 entrypoint:
 - /bin/sh
 command: [ "-c", "ocis collaboration server" ]
 environment:
 COLLABORATION_GRPC_ADDR: 0.0.0.0:9301
 COLLABORATION_HTTP_ADDR: 0.0.0.0:9300
 MICRO_REGISTRY: "nats-js-kv"
 MICRO_REGISTRY_ADDRESS: "ocis:9233"
 COLLABORATION_WOPI_SRC: https://wopiserver.owncloud.yourdomain.com
 COLLABORATION_APP_NAME: "Collabora"
 COLLABORATION_APP_ADDR: https://collabora.owncloud.yourdomain.com
 COLLABORATION_APP_ICON: https://collabora.owncloud.yourdomain.com/favicon.ico
 # COLLABORATION_APP_INSECURE: true
 # COLLABORATION_CS3API_DATAGATEWAY_INSECURE: true
 COLLABORATION_LOG_LEVEL: ${LOG_LEVEL:-info}
 volumes:
 - ./ocis-config:/etc/ocis
 logging:
 driver: local

 collabora:
 image: collabora/code:24.04.5.1.1
 ports:
 - 127.0.0.1:9980:9980
 environment:
 aliasgroup1: https://wopiserver.owncloud.yourdomain.com:443
 DONT_GEN_SSL_CERT: "YES"
 extra_params: --o:ssl.enable=false --o:ssl.termination=true --o:welcome.enable=false --o:net.frame_ancestors=owncloud.yourdomain.com
 username: admin
 password: admin
 cap_add:
 - MKNOD
 logging:
 driver: local
 restart: always
 command: ["bash", "-c", "coolconfig generate-proof-key ; /start-collabora-online.sh"]
 healthcheck:
 test: [ "CMD", "curl", "-f", "http://localhost:9980/hosting/discovery" ]

This adds two services which need to be reverse proxied, the OCIS-collaboration service which exposes port 9300 for wopiserver and Collabora which uses port 9980.

However, this is not all, we have to also modify the content security policy of ownCloud to allow Libreoffice to load inside ownCloud’s web interface.

Create a csp.yaml file and use the template prepared from ownCloud to modify the CSP policy accordingly, make sure to replace https://${COLLABORA_DOMAIN|collabora.owncloud.test}/' with the public URL of your Collabora server.

That should be it! You now have an OCIS server ready for small production use in your Homelab!

There are other services I didn’t cover like the import service to make it easier to import files from other cloud providers. You can find more templates under the ocis_full directory in ownCloud’s OCIS repo.

It’s a bit oddly organized, but each yml file is a docker-compose template, named under the service it adds. You start with OCIS.yml and the other files are the things you have to add to enable the service. Under the config directory you will find all the additional files used in the docker compose templates.

sources

https://github.com/owncloud/ocis/tree/master/deployments/examples/ocis_full

https://doc.owncloud.com/ocis/next/deployment/services/services.html

https://doc.owncloud.com/ocis/next/deployment/general/general-info.html#infinite-scale-supervised-services

How to Make Your Linux Home Server Accessible on the Internet Without Port Forwarding

Tue, 28 May 2024 00:00:00 +0000

If you want to make your home server accessible on the Internet but can’t use port forwarding (if you are behind CGNAT for example) or just aren’t comfortable having your home IP public, there are a few ways to do it.

You could just use Cloudflared, it is free, easy to set up and gives you some security protection through Cloudflare’s WAF. However, Cloudflare is a reverse proxy, meaning that your connection is decrypted, analyzed and then re-encrypted back to its origin, it also has specific T&S limits.

Herein comes Rathole, it’s a small standalone program written in Rust, it’s basically a simple TCP proxy, it doesn’t decrypt anything, it just forwards TCP traffic from one host to another using NAT traversal to avoid the need to port-forward anything.

You just need a server with a public IP to tunnel TCP traffic back to your home server.

I am going to use it to proxy Caddy, the reverse proxy I use on my home server, without MITMing any connections.

Installing Rathole

Rathole is a self-contained package, just download the one that’s compiled for your platform and run it from GitHub. You can also run it in a container.

This will be split into two parts, as Rathole needs to run on your home server and on the server with a static IP.

Public server docker-compose.yml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


services:
 rathole:
 image: rapiz1/rathole
 stdin_open: true
 tty: true
 ports:
 - 8080:8080 # api port
 - 443:443 # forwarded port
 volumes:
 - ./rathole/config.toml:/app/config.toml
 command: --server /app/config.toml

Rathole sever Config

1
2
3
4
5
6
7


# server.toml
[server]
bind_addr = "0.0.0.0:8080" # `8080` specifies the port that rathole listens for clients

[server.services.home-proxy]
token = "your_very_secure_shared_token" # Shared value between the client(your home server) and the server(the VPS)
bind_addr = "0.0.0.0:443" # port on the vps routing traffic to your home server "the client"

Home server docker setup

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


# assuming your webserver is connected to a shared docker network called web
networks:
 web:
 external: true

services:
 rathole:
 stdin_open: true
 tty: true
 volumes:
 - /home/lammah/rathole/config.toml:/app/config.toml
 image: rapiz1/rathole
 command: --client /app/config.toml
 networks:
 - web

Home server Rathole Config

1
2
3
4
5
6
7


# client.toml
[client]
remote_addr = "yourservers-ip-or-domain:8080" # The address of the server. The port must be the same with the port in `server.bind_addr`

[client.services.home-proxy]
token = "your_very_secure_shared_token" # Must be the same one on the server to pass the validation
local_addr = "caddy:443" # The address of the service that needs to be forwarded

Just start them both and you should be up and running! You will also need to make sure that the domains are pointing to the VPS correctly so that HTTPS and applications work as expected.

Source IP

There is one caveat though, and it’s kind of a big one, the source IP for all connections will be set to 127.0.0.1. That’s a big problem if you want to protect your applications from DOS or endless login attempts.

There is a solution, however, the PROXY protocol

Forward source IP with PROXY protocol in Rathole

The proxy protocol is a protocol that allows you to forward the source IP without needing too much computation.

Both the TCP proxy and the upstream application must support the PROXY protocol for this to work. Caddy supports the PROXY protocol, as a client and as a proxy.

There is an open PR for Rathole which adds support for the PROXY protocol. You can use my docker image to use the version with PROXY protocol support, or just build it locally.

Just use the same Configs as before, while adding enable_proxy_protocol = true under the [server.services.home-proxy] block. Make sure that the web server/app you’re tunneling to supports the PROXY protocol and that it’s set up correctly.

Now you should be able to get the correct source IP even with a TCP tunnel in between!

Redirect URL paths on Cloudflare pages for free

Sat, 27 Apr 2024 00:00:00 +0000

If you need to do an HTTP redirect for anything while using Cloudflare the first idea that comes to mind will be just to use Redirect Rules, however, if your goal is to redirect a path, like /old/* to /new/*, you actually can’t do that in the free plan, as regex_replace requires at least a PRO subscription.

Luckily this blog is hosted on Cloudflare pages, which has its own redirect mechanism, and you can do such redirects without needing a pro subscription.

using the _redirects file

you just need to create file called _redirects, which needs to be included in the final directory uploaded to Cloudflare pages. There are multiple site generator dependent ways to do this, I just add it manually in GitHub Actions after the site generation step.

Using Splats

The easiest way to do this is just using splats, a Splat is everything that is matched when using * after a path.

_redirects

/old/* /new/:splat 301

using Placeholders

Placeholders are another option, they do the same thing, but look cleaner I guess.

However, they have some specific limitations, so stick to splats when possible.

_redirects

/movies/:title /media/:title 302

sources

https://developers.cloudflare.com/pages/configuration/redirects/

https://community.cloudflare.com/t/transform-rule-replace-part-of-url/437813

Deploy Projects Securely Using Tailscale SSH and GitHub Actions

Sun, 31 Mar 2024 00:00:00 +0000

If you want to deploy your project to a server without exposing it or manually managing ssh keys, you can use Tailscale SSH!

Here is a quick guide to using my Github action tailscale-ssh-deploy to deploy files to a server using Tailscale SSH (not normal ssh).

Deploying a static site

For example, we will deploy a Hugo static site using this action

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36


on:
 push:
jobs:
 build-and-deploy:
 name: build and deploy
 runs-on: ubuntu-latest
 steps:
 - name: checkout
 uses: actions/checkout@v3
 with:
 fetch-depth: 0

 - name: Setup Hugo
 uses: peaceiris/actions-hugo@v2
 with:
 hugo-version: 'latest'
 extended: true

 - name: Build
 run: hugo --minify -v --gc --destination generated

 - name: Setup Tailscale
 uses: tailscale/github-action@v2
 with:
 hostname: Github-actions
 oauth-client-id: ${{ secrets.TS_OAUTH_CLIENT_ID }}
 oauth-secret: ${{ secrets.TS_OAUTH_SECRET }}
 tags: tag:ci

 - name: Start Deployment
 uses: FarisZR/tailscale-ssh-deploy@v1
 with:
 remote_host: USER@LOCAL-IPV4-ADDRESS-TAILSCALE
 directory: generated
 remote_destination: /var/www/html
 post_upload_command: systemctl restart nginx

The relevant part for us are the last two actions. First, we set up Tailscale using an Oauth client ID and secret that you generate in the settings page of your Tailnet more details.

Now make sure you specify the user and use the Tailscale IP address of the server, as I didn’t find using MagicDNS hostnames to be very reliable.

Then you just define the directory to upload to in directory and where it should be uploaded into remote_destination

You can use post_upload_command to specify a command to run after a successful upload, as an example I added a command to restart nginx.

That should do it! Now you have a workflow that automatically deploys your project to a server without exposing an ssh service or managing ssh keys!

Add Missing Linux Packages into WordPress Docker Images Without Rebuilding

Thu, 29 Feb 2024 00:00:00 +0000

If you use WordPress in docker, sometimes you may have some plugins complaining about missing libraries / packages, like for example EWWW image optimizer, that was complaining about missing image libraries.

You could of course just rebuild the image, but if you want to avoid that, you could modify the command that runs at start to install the packages you need, and then start the WordPress process:

Wordpress:apache (latest)

the default image uses Debian as a base, so we modify the command to update packages lists and then install the packages we need, and after that WordPress Entrypoint should get triggered.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


services:
 wordpress:
 image: wordpress:apache
 restart: always
+ command: sh -c "apt update && apt install -y --no-install-recommends xxxxxxx; docker-entrypoint.sh apache2-foreground"
 environment:
 WORDPRESS_DB_HOST: mariadb
 WORDPRESS_DB_USER: xxxx
 WORDPRESS_DB_NAME: wordpress
 WORDPRESS_DB_PASSWORD: xxxxx
 volumes:
 - html:/var/www/html:rw

Wordpress:fpm

Basically the same as the default Apache image, with just a different entrypoint command.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


services:
 wordpress:
 image: wordpress:fpm-alpine
 restart: always
+ command: sh -c "apt update && apt install -y --no-install-recommends xxxxxxx; docker-entrypoint.sh php-fpm"
 environment:
 WORDPRESS_DB_HOST: mariadb
 WORDPRESS_DB_USER: xxxx
 WORDPRESS_DB_NAME: wordpress
 WORDPRESS_DB_PASSWORD: xxxxx
 volumes:
 - html:/var/www/html:rw

Wordpress:fpm-alpine

As the tag implies, this image uses Alpine as a base, here we have to use apk, the package manager for alpine.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


services:
 wordpress:
 image: wordpress:fpm-alpine
 restart: always
+ command: sh -c "apk add xx xx xx; docker-entrypoint.sh php-fpm"
 environment:
 WORDPRESS_DB_HOST: mariadb
 WORDPRESS_DB_USER: xxxx
 WORDPRESS_DB_NAME: wordpress
 WORDPRESS_DB_PASSWORD: xxxxx
 volumes:
 - html:/var/www/html:rw

And now you should have the packages you need without needing to rebuild the image manually on every update!

Set up scheduled Debian/Ubuntu unattended upgrades

Wed, 31 Jan 2024 00:00:00 +0000

When you run a server that relies on containers for almost everything, keeping the system up to date has almost zero chance of breaking anything.

So here is a quick guide to setting up automatic updates on a fixed schedule + automatic reboots for kernel updates (optional)

installing unattended-upgrades

1
2


sudo apt update
sudo apt install unattended-upgrades

Configuring Unattended-upgrades

Depending on what kind of updates you feel comfortable applying, you can customize the source repository as you like.

allow security updates only

This is actually the default configuration for unattended-upgrades

/etc/apt/apt.conf.d/50unattended-upgrades

1
2
3
4
5


// "origin=Debian,codename=${distro_codename}-updates";
// "origin=Debian,codename=${distro_codename}-proposed-updates";
 "origin=Debian,codename=${distro_codename},label=Debian";
 "origin=Debian,codename=${distro_codename},label=Debian-Security";
 "origin=Debian,codename=${distro_codename}-security,label=Debian-Security";

allow all updates

/etc/apt/apt.conf.d/50unattended-upgrades

1
2
3
4
5


 "origin=Debian,codename=${distro_codename}-updates";
// "origin=Debian,codename=${distro_codename}-proposed-updates";
 "origin=Debian,codename=${distro_codename},label=Debian";
 "origin=Debian,codename=${distro_codename},label=Debian-Security";
 "origin=Debian,codename=${distro_codename}-security,label=Debian-Security";

Additional repositories

You can also automate the update for additional repositories such as Docker’s official repositories.

First, go to /var/lib/apt/lists/ and find the Docker repos filename. In this file you should find something like

label: Docker CE
Origin: Docker
Suite: bookworm

For us, the most important details here are origin and suite, as suite is the archive that unattended-upgrades needs to specify which version to use.

To add the docker repository, create a new line under the Distro’s repos in /etc/apt/apt.conf.d/50unattended-upgrades' with the following format ":";`

so in our case it would be "Docker:bookworm";, but what if we don’t want to update this every time the system is upgraded? then you use the ${distro_codename} variable.

1

"Docker:${distro_codename}";

and now unattended upgrades should automatically update packages from the docker repos!

Enable automatic reboot for kernel upgrades

If you can tolerate a little downtime, you can also automate system reboots for kernel upgrades

by changing the following settings

1

// Unattended-Upgrade::Automatic-Reboot "false

1

Unattended-Upgrade::Automatic-Reboot "true";

And you can change the reboot time by changing the default `02:00’ to any time you like (in the local server timezone). REMEMBER TO UNCOMMENT THE SETTING (remove the // and the spaces).

1

Unattended-Upgrade::Automatic-Reboot-Time "22:00";

Setting up timers

We will set two timers, one to update apt’s local repo cache and one to apply the updates.

Apt update timer

1

sudo systemctl edit apt-daily.timer

then add the following in the space between systemd comments (assuming you want to update apt lists at 00:55 server time every day)

1
2
3
4


[Timer]
OnCalendar
OnCalendar=00:55
RandomizedDelaySec=0

and then you should restart the service and check the status of the timer, it should show the next trigger time

1
2


sudo systemctl restart apt-daily.timer
sudo systemctl status apt-daily.timer

Apt upgrade timer

This is very similar to setting the update timer, we just need to make sure it’s after the update job so we can apply the latest updates as they are released.

1

sudo systemctl edit apt-daily-upgrade.timer

Again, in the blank space, add the following timer settings (to apply updates at 01:00 server time every day)

1
2
3
4


[Timer]
OnCalendar
OnCalendar=01:00
RandomizedDelaySec=0

and then you should restart the service and check the status of the timer, it should show the next trigger time

1
2


sudo systemctl restart apt-daily-upgrade.timer
sudo systemctl status apt-daily-upgrade.timer

and now you have fully automated updates for your entire system!

Sources

https://askubuntu.com/questions/87849/how-to-enable-silent-automatic-updates-for-any-repository

https://wiki.debian.org/StableProposedUpdates

https://linuxiac.com/how-to-set-up-automatic-updates-on-debian/

Scheduling Docker jobs without cron

Wed, 27 Dec 2023 00:00:00 +0000

Sometimes you need to automate some things in Docker containers, run a script or check a result online, and I honestly dread dealing with cron to do it.

That’s where Ofelia comes in! It’s a modern scheduler built from the ground up for Docker, and it interacts directly with the Docker socket to run a container, run a script in a specific container, or even on the host!

Usage with labels

I wanted to run a script to update data in a JSON file. You can configure Ofelia in several ways, but I think by far the easiest is to just use docker labels with compose.

Labels are structured like this: ofelia.<JOB_TYPE>.<JOB_NAME>.<JOB_PARAMETER>=<PARAMETER_VALUE> <JOB_TYPE>:

job-exec: this job is executed inside of a running container.

job-run: runs a command inside of a new container, using a specific image.

job-local: runs the command inside of the host running ofelia.

job-service-run: runs the command inside a new “run-once” service, for running inside a swarm

Available parameters for each job type here

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


services:
 ofelia:
 image: mcuadros/ofelia:latest
 command: daemon --docker
 volumes:
 - /var/run/docker.sock:/var/run/docker.sock:ro
 - file.json:/src/file.json
 - update_json_file.sh:/update_json_file.sh:ro
 labels:
 ofelia.job-local.update_json_file.schedule: '@every 24h'
 ofelia.job-local.update_json_file.command: '/bin/sh -c "apk add curl jq && sh /update_json_file.sh /src/file.json"'

Let me explain the setup here. First we have daemon --docker as command, because adding --docker allows us to use labels as configuration, you don’t have to do that, you can also use an INI structured file as config.

And since it’s running inside a container, and it needs access to docker to start containers and run commands, in addition to reading container labels, we also mount the docker socket at /var/run/docker.sock.

Then in the labels I defined a job to run every 24h, and the command is /bin/sh -c "apk add curl jq && sh /update_json_file.sh /src/file.json" I had to manually open a shell with /bin/sh -c to use && and run multiple commands.

And that’s it! You now have an automated Docker container without having to fiddle with cron!

Purge remote media cache in Matrix Synapse

Thu, 28 Sep 2023 00:00:00 +0000

Using the Purge Remote Media API, we can send a POST request to delete all cached remote media before a certain time:

Your account must be a server admin for this command to work.

1

curl -X POST -H "Authorization: Bearer $ADMIN_ACCESS_TOKEN" "https://matrix.org/_synapse/admin/v1/purge_media_cache?before_ts=1672527600000"

before_ts takes time in UNIX epoch milliseconds, the one in the template is at 2023-01-01 00:00. You can generate one [here] (https://currentmillis.com/)z

Generating an access token

If you don’t have an access token, you can also generate one using curl:

1
2
3


curl -XPOST \
 -d '{"type":"m.login.password", "user":"<userid>", "password":"<password>"}' \
 "https://matrix.org/_matrix/client/r0/login"

Sources

https://matrix-org.github.io/synapse/latest/admin_api/media_admin_api.html#purge-remote-media-api Bing AI.

Setup IPv6 properly on docker (Fix Source IPv6 propagation)

Tue, 26 Sep 2023 00:00:00 +0000

If you’re wondering why there are a lot of connections coming from your docker gateway right after you enabled IPv6 for your services, you’ve come to the right place.

Unfortunately, setting up IPv6 support for Docker isn’t as simple as setting up a DNS record and exposing the container to all interfaces.

If you just expose the container to an IPv6 connection without any additional setup, it will work, but without any IPv6 source propagation, you will get a lot of connections seemingly coming from the Docker gateway (usually starting with 172).

The fix for this is to make sure all networks from the bridge network to the internal docker compose networks are IPv6 ready, one network that is IPv4 only will bring the problem back.

Enable IPv6 support

/etc/docker/daemon.json

1
2
3
4


{
 "experimental": true,
 "ip6tables": true
}

This will enable ipv6tables support for Docker, which allows port mapping and network isolation, but requires the experimental flag.

Enabling IPv6 on the default bridge network

Enabling ipv6 in the config should be enough now, but to make sure all networks are IPv6 enabled, we add IPv6 to the default bridge network.

/etc/docker/daemon.json

1
2
3
4
5
6


{
 "ipv6": true,
 "fixed-cidr-v6": "2001:db8:1::/64",
 "experimental": true,
 "ip6tables": true
}

Dynamic IPv6 Subnet Allocation

Now with the previous settings you should be ready to go. However, you will need to manually set a subnet for each network, either within Compose or with docker network create --ipv6 --subnet=xxx $NETWORK_NAME.

If you want Docker to automatically allocate subnets, as it does with IPv4, you will need to provide it with a pool.

/etc/docker/daemon.json

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


{
 "ipv6": true,
 "fixed-cidr-v6": "2001:db8:1::/64",
 "experimental": true,
 "ip6tables": true,
 "default-address-pools": [
 { "base": "172.17.0.0/16", "size": 16 },
 { "base": "172.18.0.0/16", "size": 16 },
 { "base": "172.19.0.0/16", "size": 16 },
 { "base": "172.20.0.0/14", "size": 16 },
 { "base": "172.24.0.0/14", "size": 16 },
 { "base": "172.28.0.0/14", "size": 16 },
 { "base": "192.168.0.0/16", "size": 20 },
 { "base": "2001:db8::/104", "size": 112 }
 ]
}

2001:db8::/104 should not be used as a real subnet, it’s for documentation purposes only. If you want a local pool, go to simpledns.plus and replace the first two address blocks with those generated by the site.

Now restart the docker daemon:

1

sudo systemctl restart docker

Migrating Existing Containers / Docker Compose Projects

Restarting the daemon alone won’t be enough if you have containers running, especially those inside a Docker Compose project using an internal network.

You will need to remove all containers connected to a network other than bridge.

If you are using Compose, first delete all containers and their networks (this should’t delete volumes):

1

docker compose down

Add enable_ipv6: true to all networks, if you didn’t define any networks, then you are using the default network, and you should define it and add the option to it:

1
2
3


networks:
 default:
 enable_ipv6: true

Classic docker networks need to be removed and rebuilt with the --IPv6 option:

1

docker network rm $NETWORK && docker network create --ipv6 $NETWORK

Now you should have full IPv6 support with proper IPv6 source IP propagation.

Testing the changes

To make sure it actually works, run a Python container connected to a network you’ve created and see if it propagates the address or not:

1

docker run -p 8080:8080 --rm -it --network $NETWORK python:latest python3 -m http.server 8080 --bind ::

Now just connect to it on port 8080 and see if the IP is displayed correctly.

If the source still shows up as a local IPv4 address, just test it with the bridge network:

1

docker run -p 8080:8080 --rm -it python:latest python3 -m http.server 8080 --bind ::

If it works, then the problem is that the network you are using doesn’t have IPv6 enabled, if it still doesn’t work, make sure the config changes have been applied!

sources

https://docs.docker.com/config/daemon/ipv6/

setup Squid proxy with docker

Tue, 29 Aug 2023 00:00:00 +0000

I use proxies very often as a way to do split tunneling on Linux, as easy split-tunneling with WireGuard or OpenVPN isn’t there yet.

The quickest way to set up anything IMO is with docker, and here is a quick guide for squid proxy setup using the official Ubuntu-squid image, which seems to be the only up-to-date docker image on Docker hub.

docker-compose.yml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


version: "3"
services:
 proxy:
 image: ubuntu/squid
 ports:
 - "3128:3128"
 environment:
 - TZ=UTC
 volumes:
 - ./squid.conf:/etc/squid/squid.conf
 configs:
 - source: squid
 target: /etc/squid/squid.conf

 configs:
 squid:
 file: ./squid.conf

I used configs to prevent any permission issues popping up.

squid.conf

http_port 3128
http_access allow all

This is a simple setup for a local transparent proxy, it allows all connections, and it doesn’t cache or log anything.

sources

https://medium.com/setup-a-web-proxy-server-with-docker-d5c6942b5575

https://hub.docker.com/r/ubuntu/squid

How to send a matrix message using CURL and an access token

Sun, 23 Jul 2023 00:00:00 +0000

Here is a how you can send a simple matrix message with CURL:

curl 'https://matrix.org/_matrix/client/r0/rooms/!xxxxxx:example.com/send/m.room.message/?access_token=xxxxxxxx' -X PUT --data '{"msgtype":"m.text","body":"hello world"}'

generate an access token

if you don’t have an access token, you can generate one using curl too:

curl -XPOST \
-d '{"type":"m.login.password", "user":"", "password":""}' \
"https://matrix.org/_matrix/client/r0/login"

Sources

https://news.ycombinator.com/item?id=19227859 https://webapps.stackexchange.com/questions/131056/how-to-get-an-access-token-for-element-riot-matrix

setup full text search for Nextcloud on docker

Thu, 08 Jun 2023 00:00:00 +0000

Setting up Nextcloud full-text search isn’t very clear, especially with Docker, so in this post I’ll explain how to set up full-text search with Elasticsearch and Tesseract using Nextcloud’s official Docker images.

add new tesseract to your Nextcloud container

Normally we would need to create a custom image for this, but fortunately modzilla99 and Schw3pps have found a way to add additional packages without needing a custom image, by adding a custom command to the container.

latest/apache image

1

command: sh -c "apt update && apt-get install -y --no-install-recommends tesseract-ocr tesseract-ocr-eng tesseract-ocr-$(YOUR_THREE_LETTER_LANGUAGE_CODE) && /entrypoint.sh apache2-foreground"

TIP: if you want to add Cron, you don’t need a separate container, just add supervisor

1

command: sh -c "apt update && apt-get install -y --no-install-recommends tesseract-ocr tesseract-ocr-eng tesseract-ocr-$(YOUR_THREE_LETTER_LANGUAGE_CODE) && mkdir -p /var/log/supervisord && mkdir -p /var/run/supervisord supervisor && supervisord -c /supervisord.conf"

make sure to mount this file at /supervisord.conf

fpm-alpine image

1

command: -c "apk add --no-cache tesseract-ocr tesseract-ocr-data-eng tesseract-ocr-data-$(YOUR_THREE_LETTER_LANGUAGE_CODE); /entrypoint.sh php-fpm"

and if you have a separate Cron container, add this to it:

1

command: -c "apk add --no-cache tesseract-ocr tesseract-ocr-data-eng tesseract-ocr-data-$(YOUR_THREE_LETTER_LANGUAGE_CODE); /cron.sh"

Elasticsearch setup

docker-compose.yml

Add this to your Nextcloud compose file

To avoid any problems, make the Nextcloud container depends on Elasticsearch, to make sure it’s started before Nextcloud

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


services:
 elasticsearch:
 image: docker.elastic.co/elasticsearch/elasticsearch:7.17.10
 networks:
 default:
 # ports:
 # 127.0.0.1:9200:9200 #only needed if you are connecting through a docker network
 command: sh -c "bin/elasticsearch-plugin install --batch ingest-attachment; /bin/tini -s /usr/local/bin/docker-entrypoint.sh eswrapper"
 restart: always
 environment:
 - discovery.type=single-node
 - bootstrap.memory_lock=true
 - "ES_JAVA_OPTS=-Xms512m -Xmx2048m"
 user: 1000:1000
 ulimits:
 memlock:
 soft: -1
 hard: -1
 volumes:
 - elasticsearch:/usr/share/elasticsearch/data

volumes:
 elasticsearch:

Using the same trick as before, we can install the ingest-attachment plugin without the need for a custom image. There will be an error when the container is restarted as it tries to install the plugin again, but it should continue to work fine.

Nextcloud setup

Select Elasticsearch as the search platform, then add the server address. If your language requires a custom tokenizer, feel free to change it.

The other options below aren’t very important and can be configured to your liking.

Start indexing!

Test setup

1

./occ fulltextsearch:test

index all files

1

./occ fulltextsearch:index

automatic indexing

The official wiki has a section on using cron for this: https://github.com/nextcloud/fulltextsearch/wiki/Basic-Installation#live-index-service

But since we are using docker, probably with supervisord added, robeatoz has found a way to add fulltextsearch:live to the supervisord.conf file.

fulltextsearch.sh

create a new file called fulltextsearch.sh

1
2
3
4
5
6
7


#!/bin/sh
# Stop all running indexes
php /var/www/html/occ fulltextsearch:stop
# Start live index
php /var/www/html/occ fulltextsearch:live

# More information: https://github.com/nextcloud/fulltextsearch/wiki/Commands

supervisord.conf

Add this to the end of the file

1
2
3
4
5
6
7


[program:fulltextsearch_index_live]
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
user=www-data
command=/bin/sh /fulltextsearch.sh

docker-compose.yml

mount fulltextsearch.sh into the container

1
2
3


volumes:
 - ....
 - ./fulltextsearch.sh:/fulltextsearch:ro

And you are done!

sources

https://github.com/nextcloud/docker/issues/1724

Did the EU ban JSdelivr and other free CDNs?

Tue, 02 May 2023 00:00:00 +0000

So while I was updating my website, I was thinking about using JSdelivr for my blogs images, the slowest part of my website since it’s hosted on a VPS and not on Cloudflare Pages to avoid further internet centralization.

But I remembered that a court in Germany recently ruled that using Google Fonts is illegal, does this affect JSdelivr and other public CDNs? Apparently it does.

I’m not a lawyer, these are just my observations.

Data processing agreements with free CDNs

According to legalweb.io, in order to use a CDN you need to have an data processing agreement, which you don’t have with free CDNs, and the data needs to be processed within the EU or a country that is deemed to follow GDPR standards, which is certainly not the U.S.

So basically any CDN provider that you don’t have an Article 28 GDPR agreement with is illegal, any of the free CDNs like Jsdelivr are banned, that should include Bunny fonts!, which was created as a GDPR-friendly alternative to Google Fonts, but since you don’t have a DPA with them, it’s still probably illegal.

Legitimate interest and CDNs

In that LG München ruling on Google fonts, it was also mentioned that the website had no legitimate interest in using X fonts (presumably Google fonts), since the website admin could easily host the fonts themselves.

Es liegt auch kein Rechtfertigungsgrund für den Eingriff in das allgemeine Persönlichkeitsrecht vor. Ein berechtigtes Interesse der Beklagten i.S.d. Art. 6 Abs. 1 f) DS-GVO, wie von ihr behauptet, liegt nicht vor, denn X. Fonts kann durch die Beklagte auch genutzt werden, ohne dass beim Aufruf der Webseite eine Verbindung zu einem X.-Server hergestellt wird und eine Übertragung der IP-Adresse der Webseitennutzer an X. stattfindet.

What about other use cases like video hosting, which is much more bandwidth intensive and benefits a lot from CDNs? the answer won’t really matter as all your favourite streaming providers will have their own DPAs with their CDN providers, nobody’s giving you a free video CDN!

Looks like I’ll just stick to hosting my blog images on my free VPS (shout out to Oracle Free Tier), and if I really want to improve the speed of my blog, I should just move to a CDN host like Cloudflare pages.

EDIT: JSdelivr doesn’t seem to agree

JSdelivr hired a Law firm to look into this, they don’t seem to agree, but that doesn’t invalidate other opinions.

In conclusion, the ruling that has been so controversial recently does not seem to fully address the factual and technical circumstances of how jsDelivr works, and at this point as a single ruling should not lead to any real concerns about using CDN’s services. The arguments for extending to other online services a single ruling strongly emphasizing Google’s failure to adequately protect personal data are insufficient and lack substance. https://www.jsdelivr.com/blog/how-the-german-courts-ruling-on-google-fonts-affects-jsdelivr-and-why-it-is-safe-to-use/

Sources

No legitimate interest for using Google Fonts on websites, says German court

Discuss on HN!

Free Discourse S3 object storage and CDN with Cloudflare R2

Mon, 01 May 2023 00:00:00 +0000

In the official discourse S3-compatible object storage setup guide, it mentions that Cloudflare R2 is not supported, because it doesn’t handle gzipped files correctly. But Cloudflare fixed this in the 2023-03-16 update, and I tested it, and it works great on discourse.aosus.org. Even old broken gzip handling tests work now, so it looks like the problem is solved!

~~EDIT: Discourse doesn’t use the CDN URL for direct downloads, rather it uses the S3 API link, which doesn’t work for Cloudflare R2, so any attachments not embedded in the post won’t work~~ You have to enable Use S3 CDN for all uploads in the settings.

And Cloudflare R2 is free for up to 10GB! (10,000,000 reads, 1,000,000 writes), so for your average discourse forum, its probably not going to cost you anything!

setup

Follow any pre-environment step in the official setup guide

environment setup

 DISCOURSE_USE_S3: true
DISCOURSE_S3_REGION: "us-east-1" #alias to auto
#DISCOURSE_S3_INSTALL_CORS_RULE: true #it should be supported
DISCOURSE_S3_ENDPOINT: S3_API_URL
DISCOURSE_S3_ACCESS_KEY_ID: xxx
DISCOURSE_S3_SECRET_ACCESS_KEY: xxxx
DISCOURSE_S3_CDN_URL: your cdn url
DISCOURSE_S3_BUCKET: BUCKET_NAME
DISCOURSE_S3_BACKUP_BUCKET: other-private-bucket #optional
DISCOURSE_BACKUP_LOCATION: s3 #optional

I’ve been using it for a month now, and it’s working great, no problems and fantastic speed when viewing images. There is one caveat though, you can’t configure a separate S3 host for backups and the CF R2 isn’t the cheapest for storage.

Docker compose Gitops using Github Actions

Thu, 09 Mar 2023 00:00:00 +0000

So you have heard about how good GitOps is, and how much better it is than having to manage your containers by hand. But almost every time GitOps is mentioned, it means using K8S. But what about the rest of us mere mortals who are still using Docker Compose?

That’s why I started working on docker-compose-gitops-action, a GitHub action to do GitOps with Docker compose!, it should cover most edge cases for Docker compose CI deployments, swarm, uploading sidecar files, post-upload shell commands, non-exposed servers like Homelabs and more.

Server side setup

The action supports two access modes, SSH and Tailscale SSH. Tailscale SSH is particularly useful for servers behind NAT or without an exposed SSH demon, allowing you to grant SSH access without manually generating keys or exposing the server. Both Tailscale SSH and SSH require a user with access to the Docker socket i.e. without the need for sudo. This user can be different from the main user.

Repo Setup

This action can be used in two ways

Per-project directories

In this case, each project/service will have its own directory with sidecar files and docker-compose.yml.

This gives you more flexibility in configuring actions for deploying specific directories, so I think this is the way to go. And if there’s a small update, you don’t have to redeploy the whole repo.

Single compose file

If the server is only running a single project, you can put the docker-compose.yml file in the root of the repo and have the action run on every push.

Setting up the GitHub action

Before we can do anything, we need to set up the GitHub secrets for authentication. Mainly SSH private key (PEM format) + public key, or an ephemeral Tailscale API key.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36


name: deploy-project

on:
 push:
 paths:
 - 'project/**'
 - '.github/**'
 branches: [main]
 workflow_dispatch:

jobs:
 deploy:
 runs-on: ubuntu-latest
 if: github.event_name != 'pull_request'
 steps:
 - name: checkout
 uses: actions/checkout@v3
 with:
 fetch-depth: 0

 - name: Tailscale
 uses: tailscale/github-action@ce41a99162202a647a4b24c30c558a567b926709
 with:
 authkey: ${{ secrets.TAILSCALE_AUTHKEY }}
 hostname: Github-actions

 - uses: aosus/docker-compose-gitops-action@v1
 name: Remote Deployment
 with:
 remote_docker_host: root@100.107.201.124
 args: -p project up -d
 tailscale_ssh: true
 compose_file_path: project/docker-compose.yml
 upload_directory: true
 #post_upload_command: touch "upload_command.txt"
 docker_compose_directory: project

Whenever the action file is edited or the project folder is changed on the main branch, this action is triggered. I’m using Tailscale SSH, so I need to have the official Tailscale action as a pre-deploy step, to not have to specify any ssh keys or actually open any ports.

Just replace tailscale_ssh with this if you want to use pure ssh:

1
2


 ssh_public_key: ${{ secrets.SSH_KEY }}
 ssh_private_key: ${{ secrets.SSH_PRIVATE_KEY }}

Be sure to set up these secrets before running the action!

I also enabled upload_directory and set docker_compose_directory as project. The project directory will be uploaded to the server. This is useful if a container needs extra config files, and you don’t want to rebuild a custom image with the config files on every image update. It will also upload the docker-compose file since it is included in the directory.

If you need to change file permissions after uploading, you can use post_upload_command to chmod or chown files, although in most cases you will need to use sudo.

managing secrets in compose files

But what about secrets, you don’t want to upload your secret plain text into the repo right?, well you can just use GitHub secrets with environment variables!

Don’t set a value for the variable in the compose file:

environment:
- VAR

set the variable in GitHub actions

- name: Start Deployment
uses: FarisZR/docker-compose-gitops-action@v1
env:
MARIADB_PASSWORD: ${{ secrets.gnulinuxsa_wordpress_mariadb_password }}
with:
remote_docker_host: ${{ secrets.server_address }}

That should do the trick, just create an action for each of your projects, and you should have your very own Compose-powered Gitops workflow!

Credits

Photo generated by Lexica AI, and no Lexica, you can’t copyright AI generated images.

Enable Video hardware acceleration on Fedora 36+/RHEL/CentOS

Tue, 31 Jan 2023 00:00:00 +0000

(This only affects MESA, so only people using open source drivers)

Shortly before Fedora 37 release, Fedora dropped support for closed Codecs like H264, HEVC, VC1, this also includes other Red Hat distros like RHEL and CentOS stream, and probably clones like Rocky and Almalinux.

The reason is mainly to avoid Closed source software patents, Not every Distribution is affected though, as it depends on their legal jurisdiction, Ubuntu for example still includes Closed source codecs and Closed source drivers(when needed).

Enable RPMfusion

MESA Packages with closed source codecs aren’t available on the Official repos, but in RPMFusion, which is totally separate from Fedora and Red Hat, that’s why they can upload such packages.

Using the GUi

Click on the link below, and install the Free then the non-free repository matching your distribution.

https://rpmfusion.org/Configuration

using the terminal

Workstation

1

sudo dnf install https://mirrors.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm https://mirrors.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm

Silverblue

1

sudo rpm-ostree install https://mirrors.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm https://mirrors.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm

RHEL / CentOS

1
2


sudo dnf install --nogpgcheck https://dl.fedoraproject.org/pub/epel/epel-release-latest-$(rpm -E %rhel).noarch.rpm
sudo dnf install --nogpgcheck https://mirrors.rpmfusion.org/free/el/rpmfusion-free-release-$(rpm -E %rhel).noarch.rpm https://mirrors.rpmfusion.org/nonfree/el/rpmfusion-nonfree-release-$(rpm -E %rhel).noarch.rpm

CentOS Steam 8 requires an additional step

1

sudo dnf config-manager --enable powertools

RHEL 8 also requires an additional step

1

sudo subscription-manager repos --enable "codeready-builder-for-rhel-8-$(uname -m)-rpms"

Install Drivers with closed source codecs included

AMD

1
2


sudo dnf swap mesa-va-drivers mesa-va-drivers-freeworld
sudo dnf swap mesa-vdpau-drivers mesa-vdpau-drivers-freeworld

If using i686 compat libraries (for steam or alike):

1
2


sudo dnf swap mesa-va-drivers.i686 mesa-va-drivers-freeworld.i686
sudo dnf swap mesa-vdpau-drivers.i686 mesa-vdpau-drivers-freeworld.i686

Intel

1

sudo dnf install intel-media-driver

not so recent hardware

1

sudo dnf install libva-intel-driver

Nvidia

Nvidia’s driver doesn’t support VAAPI, so we need to use a bridge to translate VAAPI calls into NVDEC/NVENC

1

sudo dnf install nvidia-vaapi-driver

Credits

https://ask.fedoraproject.org/t/proprietary-video-codecs-are-no-longer-hardware-accelerated-by-default-on-amd-gpus-on-fedora-37/28965

https://rpmfusion.org/Howto/Multimedia

Photo by Thomas Foster on Unsplash

setup a Samba share with network discovery using docker-compose

Sat, 31 Dec 2022 00:00:00 +0000

I am going to show you how to quickly set up a discoverable SMB share, it’s much easier than you think! Massive thanks to crazymax for his awesome docker containers that make this possible.

we’re going to use crazymax’s smb container, pretty easy to setup and docker native. its recommended to use host networking for it.

docker-compose.yml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


## https://fariszr.com/en/docker-smb-network-discovery/
version: "3.5"

services:
 samba:
 image: crazymax/samba
 container_name: samba
 network_mode: host
 volumes:
 - ./smb:/data
 - ./downloads:/downloads
 environment:
 - "TZ=$TIMEZONE"
 - "SAMBA_LOG_LEVEL=0"
 restart: always

don’t forget to change $TIMEZONE to your local timezone, list of tz time zones

Configuration

Unlike normal SMBd, the container uses YAML for configuration.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


auth:
 - user: root
 group: root
 uid: 0
 gid: 0
 password: bar

global:
 - "force user = root"
 - "force group = root"

share:
 - name: downloads
 path: /downloads
 browsable: yes
 readonly: no
 guestok: yes
 veto: no
 recycle: yes 

Here is a basic config for an open local share. All my files are owned by root, that’s why i need to force root as the default user.

Put the config in the mounted data folder, so in the case of compose file above, its ./smb/config.yml

You can add more shares or more users and far more in the config. More details about docker-smb’s configuration options are in the project’s GitHub repo

After this you should be able to use the SMB share, though not discoverable.

WSDD, SMB network discovery for Windows

Add this to the docker-compose.yml file

1
2
3
4
5
6


 wsdd:
 image: jonasped/wsdd
 network_mode: host
 environment:
 - 'HOSTNAME=$HOSTNAME'
 restart: always

just replace $HOSTNAME with your hostname and then start the container

1

docker compose up -d

Now you should see the share with the $HOSTNAME as its name in the network tab (if local device discovery is enabled) in Windows file explorer.

Avahi, SMB network discovery for MacOS and Linux

Add this to docker-compose.yml, make sure to change $HOSTNAME to whatever you want the share’s name to be.

1
2
3
4
5
6
7


 avahi:
 image: ydkn/avahi
 hostname: $HOSTNAME
 network_mode: host
 volumes:
 - ./avahi-services:/etc/avahi/services:ro
 restart: always

avahi-services/smb.service

create the file smb.service with the following content:

1
2
3
4
5
6
7
8
9


<?xml version="1.0" standalone='no'?>
<!DOCTYPE service-group SYSTEM "avahi-service.dtd">
<service-group>
 <name replace-wildcards="yes">%h</name>
 <service>
 <type>_smb._tcp</type>
 <port>445</port>
 </service>
</service-group>

save it and start the container

1

docker compose up -d

You should now have Network discovery working on almost every OS!

Sources

https://github.com/crazy-max/docker-samba/issues/1

https://github.com/crazy-max/docker-samba

Setup Cloudflared using Docker without the ZT dashboard

Fri, 28 Oct 2022 00:00:00 +0000

I wanted to set up Cloudflared, but I couldn’t find anything about setting it up in docker, especially without the Zero Trust dashboard (because it kept refusing my credit card for some reason). So here it is!

I am aiming to set up one tunnel per container, which I think is better and easier to manage than multiple tunnels in one Cloudflared instance.

docker-compose.yaml

version: '3.1'
services:
cloudflared:
image: cloudflare/cloudflared
restart: always
environment:
- TUNNEL_ORIGIN_CERT=/etc/cloudflared/cert.pem
- TUNNEL_EDGE_IP_VERSION=auto #use IPv4 and IPv6
volumes:
- ./cloudflared:/etc/cloudflared
command: tunnel run YOUR_TUNNEL_NAME

Linking Cloudflared with your domain

Create `cloudflared` dir

firstly you need to create the ./cloudflared directory before running any docker commands, because on container start up It’s going to create the directory as root, and Cloudflared runs as the distroless nonroot(id 65532) user, so you will just end up with permission problems.

mkdir ./cloudflared

Rootful

sudo chown -R 65532:65532 ./cloudflared

Rootless

this is assuming your subid and subgid ranges are 100000:65536

sudo chown -R 165531:165531 ./cloudflared

User name spaces remapping

for some reason, docker in UserNS mode uses different IDs, even though I am using the same subuid/subgid.

sudo chown -R 165532:165532 ./cloudflared

docker run -v $PWD/cloudflared:/home/nonroot/.cloudflared cloudflare/cloudflared login

Open the link in your browser and select which domain you would like to use, and then it will generate the origin certificate.

Create a new cloudflared tunnel

docker run -v $PWD/cloudflared:/etc/cloudflared cloudflare/cloudflared tunnel create YOUR_TUNNEL_NAME

We switched from /home/nonroot/.cloudflared to /etc/cloudflared because tunnel files are generated in the /etc directory. We overrode the default certificate location in the compose file using the TUNNEL_ORIGIN_CERT variable.

Now you will find in ./cloudflared a cert.pem file and a .json file, the name of the file is your Tunnel ID. Copy the tunnels ID and replace YOUR_TUNNEL_ID with it in the following steps.

Configuring the tunnel

create a config.yml file inside the ./cloudflared directory

This is a basic configuration for a WordPress site inside the same docker network as Cloudflared, running on port 80.

tunnel: YOUR_TUNNEL_ID
credentials-file: /etc/cloudflared/YOUR_TUNNEL_ID.json
ingress:
- hostname: domain.tld
service: http://wordpress:80
- hostname: www.domain.tld # otherwise its going to end up in a 404
service: http://wordpress:80
- service: http_status:404 # any other domain will result in a 404

You could further customize the configuration to your liking.

here are the details about it.

But this would be enough for most setups.

Start it up!

Make sure cloudflared is running in the same network as your other container if you are using DNS hostnames, and it should just work!

docker compose up -d

DNS Records

Add a CNAME record to your domains pointing to YOUR_TUNNEL_ID.cfargotunnel.com, and make sure to enable Cloudflares proxy (the cloud needs to be orange).

Rootless/UserNs note

if you see this error

WRN The user running cloudflared process has a GID (group ID) that is not within ping_group_range. You might need to add that user to a group within that range, or instead update the range to encompass a group the user is already in by modifying /proc/sys/net/ipv4/ping_group_range. Otherwise cloudflared will not be able to ping this network error="Group ID 65532 is not between ping group 65534 to 65534"
WRN ICMP proxy feature is disabled error="cannot create ICMPv4 proxy: Group ID 65532 is not between ping group 65534 to 65534 nor ICMPv6 proxy: socket: permission denied"

Currently, This has no effect, as Cloudflared still doesn’t support ICMP over QUIC anyway. I tried fixing it by setting net.ipv4.ping_group_range = 0 2147483647, but it still didn’t work, so just ignore it for now.

If you have a solution, write it in the comments!

How to set "Cache-Control" Header for static files in Caddy

Fri, 30 Sep 2022 00:00:00 +0000

If you are seeing the “Serve static assets with an efficient cache policy” notice in PageSpeed, you need to add the Cache-Control header to your website.

Here is how to do it if you are using Caddy.

Cache-Control

To set a Cache-Control header only for your static files like, .js, .jpg, and others, just add this to the relevant part of your Caddyfile(under the site block if the Caddyfile includes multiple websites)

@static {
file
path *.ico *.css *.js *.gif *.webp *.avif *.jpg *.jpeg *.png *.svg *.woff *.woff2
}
header @static Cache-Control max-age=5184000

The files are going to be cached for 5184000 seconds, so 60 days, you can change it if you want.

Source

https://caddy.community/t/correct-way-to-set-expires-on-caddy-2/7914/13

How to compress static site to Brotli and Gzip

Wed, 14 Sep 2022 00:00:00 +0000

If you deploy your static site to a Web server like Caddy, you might need to pre-compress it, especially if you want to use Brotli with Caddy as it’s only supported for pre-compressed assets.

Brotli

find www.new -type f \( -name '*.html' -o -name '*.js' -o -name '*.css' -o -name '*.xml' -o -name '*.svg' \) \
-exec /bin/sh -c 'brotli -q 11 -o "$1.br" "$1"' /bin/sh {} \;

Gzip

find www.new -type f \( -name '*.html' -o -name '*.js' -o -name '*.css' -o -name '*.xml' -o -name '*.svg' \) \
-exec /bin/sh -c 'gzip -v -f -9 -c "$1" > "$1.gz"' /bin/sh {} \;

Source

https://github.com/maruel/hugo-tidy/blob/main/docker-entrypoint.sh#L49

how to fix screen sharing in Chromium browsers on Wayland

Fri, 02 Sep 2022 00:00:00 +0000

If you use a distro with Wayland, you might have noticed, that screen sharing might not work, it will probably show a black screen, instead of your display

Why

This is because Wayland doesn’t give the permission for a program to capture the screen the same way X.ORG does.

Since your distro/Desktop environment uses Wayland, it probably uses Pipwire too, we will use it to share our screen.

Make Chromium use Pipewire

open your chromium browser and open this link chrome://flags/#enable-webrtc-pipewire-capturer

And Then just restart your browser from the restart option, and you should be OK.

Notes

It’s still not seamless though, when you share your screen, there will be two pop-ups instead of only one. One from the browser and then one from Pipewire.

And when you select your screen, It’s going to prompt you to select it again when its actually going to share it to the application, and not just preview it.

As for Electron apps, this Option should be enabled by default on new version, however on old ones, you might need to find a way to enable it manually.

But most electron apps have a web version anyway, so just use it instead of the app!

About

Mon, 01 Jan 0001 00:00:00 +0000

Hi, I am Faris, passionate about Free and Open source software, and Cloud computing. I work on the Aosus Community, the biggest Arabic community for FOSS, and I created ARhackintosh, a place where you found anything hackintosh related in Arabic.

I spend my time experimenting with FOSS and cloud computing as I like to learn by doing.

My Projects

Mon, 01 Jan 0001 00:00:00 +0000

The Aosus Community

Aosus is the biggest Arabic community for FOSS, we want to spread Free and open source software in the Arab world, and spread the culture around contributing to FOSS.

I Joined Aosus in 2021 to work on a new re-launch for the project, first by merging it with another Linux telegram chat group that had more than 3000 members.

Then I worked on redesigned the community page, and launching the new official website and blog, of course all of this came with a complete overhaul of Aosus’s Technical infrastructure, now fully dockerized.

I also launched Aosus services, where aosus hosts Privacy focused interfaces/proxies for popular websites, like YouTube, Twitter, Reddit etc. without tracking or Ads.

I also managed The Aosus Writing Contest summary in English, the first Contest of its kind in the Arabic world to support writers that write about FOSS in Arabic. You can find ways to support Aosus here, you can also contribute to Aosus’s projects

my Achievements in Aosus

(unfortunately most links are only in Arabic, it’s an Arabic focused project after all)

Create a page about the Community ant its projects (partially translated)
Creating a Matrix server for aosus
Redesign the Discourse forum, you can see the old designs on the Internet Archive
launching the Aosus writing contest, the first Contest of its kind in the Arabic world.
Aosus joining Open Collective, Aosus was one of the first Arabic to join the platform, to have financial transparency and independence.
Launch Aosus services, to protect user’s digital privacy
create a plan for the upcoming project, Torjoman, a new way to crowdsource translations directly from the users favorite apps. (English description available)
Add Aosus to new social media platforms
Join Discord using matrix bridges
Getting non-profit 501(c)(3) status in the U.S. with Aosus’s new fiscal sponsor, HCB from Hack club.

ARhackintosh

ARhackintosh, is the first complete Arabic community for Hackintosh, with an Open source guide, based on dortania, and a Kext archive.

If you know Arabic, ARhackintosh needs new maintainers, if you’re interested more details here

(Yes that’s an actual word mark, people can read it)

I started ARhackintosh in 2020, to create a dedicated place for All things Hackintosh in Arabic.

It helped massively in discovering my passion and love for Linux and technical infrastructure, and got me into the world of git and FOSS.

ARtutorial

the biggest part of the project, and the one where I spent most of the dev time is ARtutorial. More than 4000 words, and 1250+ commits, the biggest Arabic guide for how to install Hackintosh. Its completely open source

I tried to keep the guide simple, but also not hide any real complexities of installing a Hackintosh. Because its actually Complex, hiding it from the user always ends up in lost data, or lost time.

Kext Archive

Kext archive aims to collect all crucial Kexts(Kernel extensions/basically macOS Drivers), with direct links to developers repo and downloads, with handwritten descriptions in Arabic, and clear categories for kexts.

to my knowledge, It’s still somewhat unique, as I haven’t seen anything similar in English.

The archive aims to fix the hassle of finding drivers, and finding an up-to-date version of it. Most Hackintosh guides in Arabic are outdated video guides, with no updates or any kind of edits from the creator. Kext archive aims to create a simple place where users can find advanced driver/kexts for their devices. It now has more than 50 drivers listed.

Search

Mon, 01 Jan 0001 00:00:00 +0000