Fossil SCM and Concourse

I while I think builds.sr.ht is an excellent service, it doesn’t give a me lot of freedom when it comes to hosting and VCS: it forces me to use Sourcehut and Git, both of which I find OK; but I love the freedom and ease of use that Fossil provides, packed with tons of features.

Software configuration management

Git is good because everyone uses it; but in terms of ease of use and features, it is not necessarily the best. Generally, you can’t expect to run, manage and maintain an entire project with Git alone: you will need to choose a development platform like GitHub or Sourcehut, or self-host CGit or Gitea. Most of those solutions won’t give you a lot of freedom and control, and they will lock you into their ecosystems.

Fossil is a distributed, no-nonsense, self-contained, easy to use and powerful software configuration management system (SCM), with integrated bug tracking, wiki, forum and technotes. You can expect expect to run, manage and maintain an entire project with Fossil alone… easily. It comes with an awesome web interface that you can even run locally. Think of it like a “GitHub-in-a-box” that you can use even locally and without internet, and doesn’t require installation (only a tiny standalone binary).

The problem with Fossil is its lack of integrations and tooling, but that’s also a feature! It is hard to integrate complex functionalities, such as CI/CD and automation, in a fully distributed way. But, instead of integrating Fossil with existing tools, why not integrating existing tools with Fossil?

Once I fell in love with Fossil, I thought of moving my projects to my own server, but the lack of continuous deployment services was stopping me from making the switch, since builds.sr.ht was already essential to me. But I wouldn’t let that stop me! I decided I should have full control over my own projects, so I started searching for alternatives I could control, and hosted my own Fossil server.

Continuous thing-doing

I needed a self-hosted, flexible, easy to use and SCM-independent continuous thing-doer. First, I considered Git-exporting my Fossil projects so I could use builds.sr.ht indirectly; but that wouldn’t be fun, and I didn’t want to rely on Sourcehut or Git whatsoever.

Jenkins

I tried it first, but didn’t like it, because builds don’t run in containers by default, some plugins are too old and don’t work anymore, and Docker integration was a pain in the ass; so I gave up, it didn’t suit my (simple) needs, it is awful!

Concourse

Then, I came across Concourse, which turned out to be exactly what I needed! It works exactly how I want by default, has a nice CLI called fly, config is written in YAML and it is extremely flexible, and everything is done inside containers!

My automated workflow

Note: an improved version of this workflow is explained here, using the Concourse resource for Fossil I wrote.

Warning: my explanations are probably vague and incorrect, but that’s because my experience on Sysadmin and DevOps is still limited, I’m kind of a newbie.

Warning: this may (probably) not be the best and most secure configuration, also because I’m a newbie. I would appreciate feedback and suggestions!

This blog, along with some of my projects and websites, are now hosted in my own Fossil server, served with the built-in HTTP server, reverse-proxied with NGINX and secured with Certbot. The repository list is a static page I made. I keep the server running with Supervisor, with the following command: (only files ending in .fossil will be served)

fossil server /var/www/fossil/ --https --port 8079

Then, I have a Concourse instance running in a Docker container. The configuration for each pipeline is written in YAML, and deployed via the CLI. The current pipeline.yml for this blog, for example, contains something like this, some stuff is omitted for brevity:

resource_types:
  - name: rss
    type: docker-image
    source:
      repository: suhlig/concourse-rss-resource
      tag: latest
        
resources:
  - name: fossil
    type: rss
    source:
      url: https://fossil.avalos.me/avalos-indie-blog/timeline.rss?tag=trunk

jobs:
  - name: deploy-to-server
    serial: true
    plan:
      - get: fossil
        trigger: true
      - task: build
        config:
          platform: linux
          image_resource:
            type: docker-image
            source: { repository: debian, tag: buster-slim }
          params:
            ...
          outputs:
            - name: static
          run:
            path: /bin/bash
            args:
              - -c
              - |
                ...
      - task: deploy
        config:
          platform: linux
          image_resource:
            type: docker-image
            source: { repository: alpine, tag: edge }
          inputs:
            - name: static
          params:
            AWS_ACCESS_KEY_ID: ((AWS_ACCESS_KEY_ID))
            AWS_SECRET_ACCESS_KEY: ((AWS_SECRET_ACCESS_KEY))
            ...
          run:
            path: /bin/sh
            args:
              - -c
              - |
                ...

First, I set a resource called fossil, from an existing resource_type that fetches a RSS feed, Concourse will automatically trigger a new build job every time this resource changes (i.e. there’s a new RSS entry) (i.e. there’s a new commit).

Then, I set a single job called deploy-to-server, with a plan consisting on a series of tasks. The first task, get, will be triggered by fossil. The second is build, which runs a series of commands inside a debian:buster-slim container (including checking-out from Fossil and compiling with Hugo), and produces an output called static (into a mounted folder with the same name). Once the task is generated, the task deploy will mount static as an input to a alpine:edge container that will deploy the generated files.

The deployment is done securely via AWS EC2 Instance Connect (needs to be configured on the instance you want to access), a new SSH key pair is generated in the container, the public key is sent to AWS (using a restricted IAM role), which is only available for a short period of time. Then, the actual deployment is done via rsync, into the public folder.

The params of both build and deploy will be passed as environment variables to each container, and as you may notice, there are (()) on the params of deploy, why? Because those are variables I load from a vars.yml file—which in this case contains some secrets—that gets encrypted on the server once deployed, and decrypted only for a short time when used (I think). You can also load secrets from Vault or external secret managers, but you may need extra config.

Finally, the command I use to deploy my pipeline to Concourse is:

fly set-pipeline \
    -t ci.avalos.me \
    -p blog.avalos.me \
    -c pipeline.yml \
    -l vars.yml

“Why Fossil, may I ask?”

The main reason is: because I just felt like it… that’s basically it. Also because it gives me a lot of control, convenience and features, but that’s not necessarily what drove me into Fossil initially. I don’t have any problem with Git, it is just that I’ve always felt like using non-mainstream stuff, probably as the result of a need for identity and uniqueness, an urge to be different… is that wrong? Am I wrong? Is it a bug or a feature in me? Probably both, I’m not sure. I don’t use things because they are useful to me, most of the time I use them because… aesthetics.