How does this blog work?

I've often spoken to people about the inner workings of this blog, teasing elements and snippets to those who ask, but I've yet to actually go into detail about how everything comes together to present the very page you're seeing right now, or the post list which contains it. Let's change that.

The dark days

Some of you may remember my old blog website. That hunk of junk was powered by WordPress, served by a shared hosting provider to whom I'm no longer a customer. It served its time, and it worked, but it simply wasn't scalable. I needed an actual website, so I thought “why not implement the blog part of it too?” - and so this site was born. In case you were wondering, this site is an ASP.NET Core app and it's even source available. I am reluctant to use the term “open source” because I have yet to decide on the license which I feel is appropriate, however I have no issues with people reviewing my codebase for educational purposes (I encourage you to dig deep and learn how it works!) or to notify me of any bugs you come across in my implementation so I can roll out a fix.

Anyway. Yes. The site. The first thing I needed to do was identify my requirements.

The requirements

Backwards compatibility

I've shared permalinks to my blog posts around the internet and on Discord. It was of utmost priority to ensure that those links remain valid indefinitely. For example, my post titled “Shorter code ≠ better code” had the permalink https://blog.oliverbooth.dev/2022/05/11/shorter-code-is-not-better-code - this link has to work for as long as this site is live.

But there's a problem with that. The root domain oliverbooth.dev and the blog subdomain blog.oliverbooth.dev would have to be served by different sites - different applications. Either that or I set up some nginx reverse proxy nonsense, configure ASP.NET Core to accept two hosts, and maintain two distinct SSL certificates for what is essentially one site. This new site doesn't use a subdomain. Instead, my blog is available at the /blog endpoint, i.e. https://oliverbooth.dev/blog.

This is easy enough to account for, however. All I had to do was configure nginx to return HTTP 301 and pass it the same request URI that was given.

server {
  server_name blog.oliverbooth.dev ;

  add_header Strict-Transport-Security max-age=31536000;

  location / {
    proxy_pass https://localhost:2845/blog;

    proxy_set_header HOST $host;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

    return 301 https://oliverbooth.dev/blog$request_uri;
  }
}

This now means https://blog.oliverbooth.dev/2022/05/11/shorter-code-is-not-better-code will redirect to https://oliverbooth.dev/blog/2022/05/11/shorter-code-is-not-better-code and everything is dandy.

It has to be Markdown

WordPress has a fairly easy-to-use WYISWYG editor, but the idiosyncrasies in its implementation means it actually saves as HTML and litters it with a ton of boilerplate needed for that editor to function properly. Take a look at how a paragraph is rendered in WordPress:

<!-- wp:paragraph -->
<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Integer feugiat tempor finibus. Fusce id molestie nunc. Sed erat neque, lacinia id sollicitudin ut, mollis sit amet tellus. Vivamus tincidunt blandit viverra. Vestibulum tincidunt tristique finibus. In in lectus lacinia, condimentum nulla vel, eleifend nibh. Aliquam sapien mauris, consequat vitae ipsum placerat, vehicula aliquet sapien. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec faucibus pretium purus vitae faucibus. Nullam sed malesuada orci. Quisque vulputate, ex vel eleifend scelerisque, sem neque venenatis ligula, in porttitor mi eros at urna. Fusce vitae mollis sem. Nam sit amet convallis magna. Curabitur commodo magna mauris, non condimentum leo iaculis fringilla. Sed varius libero non dui vehicula viverra. Aenean efficitur hendrerit facilisis.</p>
<!-- /wp:paragraph -->

It actually saves with the <p> tags, and goes a step further by adding this weird, superfluous <!-- wp:paragraph --> framing. Thanks I sincerely hate it. I want my blog system to support Markdown. That way, I can write my posts in Obsidian or Notion or hell, even GitHub's editor, and it should look more or less the same.

After a little digging, I came across Markdig - a “CommonMark compliant, extensible Markdown processor for .NET”. It fits my needs perfectly, and the keyword in that elevator pitch is “extensible”. This proved to be essential for what I need, but I'll talk about that later in this post.

After installing Markdig to the project, it means every single one of my posts can be converted to Markdown - a mostly automatic process with just a little Regex magic to remove WordPress boilerplate and translate HTML to Markdown where applicable.

IL codeblocks

When my site was WordPress, I used HighlightJS to render codeblocks. Now don't get me wrong, HightlightJS is decent and it's certainly a very popular option. However, it doesn't have built in support for IL/CIL/MSIL - the readable version of what .NET languages like C# compile to. This was important, as I actually have a post which contains some IL.

Of course, one could spend some time developing a plugin or defining the language rules and keywords to add support for it. Sadly in my search, I found no such plugin or definition and I didn't fancy writing one myself. All hope is not lost though, I eventually stumbled on an alternative to HighlightJS named PrismJS. It supports 297 languages as of the time of writing, compared to HightlightJS with only 192. One of those 297 is CIL, exactly what I need. Go ahead, take 'er for a spin:

.class public auto ansi beforefieldinit
  ConstructorTest
    extends [UnityEngine.CoreModule]UnityEngine.MonoBehaviour
{

  .field private bool _isAlive
    .custom instance void [UnityEngine.CoreModule]UnityEngine.SerializeField::.ctor()
      = (01 00 00 00 )

  .method public hidebysig specialname rtspecialname instance void
    .ctor() cil managed
  {
    .maxstack 8

    // [5 22 - 5 51]
    IL_0000: ldarg.0      // this
    IL_0001: ldc.i4.1
    IL_0002: stfld        bool ConstructorTest::_isAlive
    IL_0007: ldarg.0      // this
    IL_0008: call         instance void [UnityEngine.CoreModule]UnityEngine.MonoBehaviour::.ctor()
    IL_000d: nop
    IL_000e: ret

  } // end of method ConstructorTest::.ctor
} // end of class ConstructorTest

With that out of the way…

TeX rendering

In my tutorial What is delta time?, I display an equation:

This is written in TeX. If you're unfamiliar with TeX, it's a subset specification of LaTeX - essentially a word processor on steroids.¹ If you've ever come across research papers or theses, and noticed they all seem to have that distinctive structure with the serif font and equations dotted around, you'll know LaTeX.

TeX is the part which is responsible for rendering the equations. It uses a markup style language to denote symbols and layout. For example, that equation above was written like so:

$$
\frac{2}{\frac{1}{\Delta t}} = 2 \Delta t
$$

I did try to find a .NET plugin for this, however I ultimately decided to do it client-side using the package KaTeX. It's not perfect, but it certainly satisfies my needs. I may change it in future though so we'll see what happens.

So why Markdig?

It's extensible, of course! Why does that matter? Well, it means I can define my own additional flavour of Markdown. Specifically, it allows me to implement a system that resembles Wikipedia-style templates. One use case of this you've already seen in this post is the image above! Every image on every post on my blog contains an quasi-related caption that serves no real purpose except for adding a bit of humour to an otherwise verbose post. That image isn't rendered using the Markdown syntax for an image. ![alt text](image.jpg) only lets you define the “alt” attribute, and you have to specify the full absolute URL of the image in question. If the idea is to keep this site future-proof in the event that I ever decide to change domains, having to update the full URL to every image on every post would be a fucking nightmare.

That image is, in fact, written in source like so:

{{Image:Blog|2024-11-04|sample-latex.png|I wish I could submit university coursework using this, but nOoOoO we have to submit a .docx so the automatic word counter works.}}

Another example you may have seen from other posts is callouts like “Guest Post” or “Legacy Post”. All I have to do is write:

{{GuestPost}}

And I get this result:

So how does this work? How do my templates get translated to HTML?

These templates are defined in the database that powers this website. I have a template named “Image”, with a variant “Blog”, whose entry looks like so:

<figure class="figure text-cener" style="display: block;">
	<img class="figure-img img-fluid" src="https://cdn.olivr.me/blog/img/{ArgumentList[0]:date:yyyy/MM}/{ArgumentList[1]}">
	<figcaption class="figure-caption text-center">{ArgumentList[2]:markdown:}</figcaption>
</figure>

Take note of the variable named ArgumentList - we'll get to this in a moment.

Markdig allows you to select the Markdown components you need for your rendering. So if you only want your Markdown renderer to support footnotes, you might have:

builder.Services.AddSingleton(provider => new MarkdownPipelineBuilder()
    .UseFootnotes()
    .Build());

But I made my own extension:

// ...
    builder.Use(new TemplateExtension(templateService));
// ...

TemplateExtension is a custom Markdig extension, defined like so:

public sealed class TemplateExtension : IMarkdownExtension
{
    private readonly ITemplateService _templateService;

    public TemplateExtension(ITemplateService templateService)
    {
        _templateService = templateService;
    }

    public void Setup(MarkdownPipelineBuilder pipeline)
    {
        pipeline.InlineParsers.AddIfNotAlready<TemplateInlineParser>();
    }

    public void Setup(MarkdownPipeline pipeline, IMarkdownRenderer renderer)
    {
        if (renderer is HtmlRenderer htmlRenderer)
        {
            htmlRenderer.ObjectRenderers.Add(new TemplateRenderer(_templateService));
        }
    }
}

Okay, so this extension is composed of a “parser” which consumes the original input (for example {{GuestPost}}), and a “renderer” which outputs the HTML result. I'm not going to go too deep into how TemplateInlineParser works, but you are free to examine the source and see for yourself here. Essentially it consumes a block until it encounters what it determines to be a “template” in the form {{Name[:Variant][|Argument1|Argument2|...]}}. This class then constructs a TemplateInline which encapsulates the name, the variant (if any), and the argument list:

public sealed class TemplateInline : Inline
{
    public string ArgumentString { get; set; } = string.Empty;

    public IReadOnlyList<string> ArgumentList { get; set; } = ArraySegment<string>.Empty;

    public string Name { get; set; } = string.Empty;

    public IReadOnlyDictionary<string, string> Params { get; set; } = null!;

    public string Variant { get; set; } = string.Empty;
}

So using the Image template mentioned above, we have the name Image, the variant Blog, and the ArgumentList which would contain the following values:

[
	"2024-11-04",
	"sample-latex.png",
	"I wish I could submit university coursework using this, but nOoOoO we have to submit a .docx so the automatic word counter works."
]

Next, TemplateRenderer kicks in and is told to convert the TemplateInline object into the output HTML:

internal sealed class TemplateRenderer : HtmlObjectRenderer<TemplateInline>
{
    private readonly ITemplateService _templateService;

    public TemplateRenderer(ITemplateService templateService)
    {
        _templateService = templateService;
    }

    protected override void Write(HtmlRenderer renderer, TemplateInline template)
    {
        renderer.Write(_templateService.RenderGlobalTemplate(template));
    }
}

Inside the template service we have the method being called, RenderGlobalTemplate. This method accepts the TemplateInline object and performs a DB query on the name to find the template's HTML result:

public string RenderGlobalTemplate(TemplateInline templateInline)
{
    if (templateInline is null)
    {
        _logger.LogWarning("Attempting to render null inline template!");
        throw new ArgumentNullException(nameof(templateInline));
    }

    _logger.LogDebug("Inline name is {Name}", templateInline.Name);
    if (_customTemplateRendererOverrides.TryGetValue(templateInline.Name, out CustomTemplateRenderer? renderer))
    {
        _logger.LogDebug("This matches renderer {Name}", renderer.GetType().Name);
        return renderer.Render(templateInline);
    }

    return TryGetTemplate(templateInline.Name, templateInline.Variant, out ITemplate? template)
        ? RenderTemplate(templateInline, template)
        : GetDefaultRender(templateInline);
}

Now here's where the magic really happens. If a template is found, which in this case it is, we call RenderTemplate passing in the received TemplateInline as well as the ITemplate it fetched from the database.

Inside this method I construct an anonymous object that houses - most critically - the ArgumentList. This object is then passed as an argument to SmartFormat's Format method:

public string RenderTemplate(TemplateInline templateInline, ITemplate? template)
{
    if (template is null)
    {
        return GetDefaultRender(templateInline);
    }

    Span<byte> randomBytes = stackalloc byte[20];
    Random.NextBytes(randomBytes);

<mark>    var formatted = new</mark>
<mark>    {</mark>
<mark>        templateInline.ArgumentList,</mark>
<mark>        templateInline.ArgumentString,</mark>
<mark>        templateInline.Params,</mark>
<mark>        RandomInt = BinaryPrimitives.ReadInt32LittleEndian(randomBytes[..4]),</mark>
<mark>        RandomGuid = new Guid(randomBytes[4..]).ToString("N"),</mark>
<mark>    };</mark>

    try
    {
        return <mark>_formatter.Format(template.FormatString, formatted)</mark>;
    }
    catch
    {
        return GetDefaultRender(templateInline);
    }
}

Remember in the HTML snippet, how there included a usage of ArgumentList?

<figure class="figure text-cener" style="display: block;">
	<img class="figure-img img-fluid" src="https://cdn.olivr.me/blog/img/{ArgumentList[0]:date:yyyy/MM}/{ArgumentList[1]}">
	<figcaption class="figure-caption text-center">{ArgumentList[2]:markdown:}</figcaption>
</figure>

SmartFormat is actually using the values from the object, specifically templateInline.ArgumentList which was initially populated by the TemplateInlineParser. We're not done yet, however. For this to work completely, I added a custom formatter to SmartFormat called “date” (hence the ArgumentList[0]:date:yyyy/MM)

public sealed class DateFormatter : IFormatter
{
    public bool CanAutoDetect { get; set; } = true;

    public string Name { get; set; } = "date";

    public bool TryEvaluateFormat(IFormattingInfo formattingInfo)
    {
        if (formattingInfo.CurrentValue is not string value)
            return false;

        if (!DateTime.TryParseExact(value, "yyyy-MM-dd",
                CultureInfo.InvariantCulture,
                DateTimeStyles.None,
                out DateTime date))
            return false;

        formattingInfo.Write(date.ToString(formattingInfo.Format?.ToString(), CultureInfo.InvariantCulture));
        return true;
    }
}

This converts the source string 2024-11-04 to 2024/11, coupled with the filename sample-latex.png, giving the full URL https://cdn.olivr.me/blog/img/2024/11/sample-latex.png - the URL of the image. The caption is then taken from ArgumentList[2], and yet another formatter called “markdown” is executed and dumped into a <figcaption>. The “markdown” formatter simply runs the entire Markdig pipeline again, recursively.

public sealed class MarkdownFormatter : IFormatter
{
    private readonly IServiceProvider _serviceProvider;

    public MarkdownFormatter(IServiceProvider serviceProvider)
    {
        _serviceProvider = serviceProvider;
    }

    public bool CanAutoDetect { get; set; } = true;

    public string Name { get; set; } = "markdown";

    public bool TryEvaluateFormat(IFormattingInfo formattingInfo)
    {
        if (formattingInfo.CurrentValue is not string value)
            return false;

        var pipeline = _serviceProvider.GetService<MarkdownPipeline>();
        formattingInfo.Write(Markdown.ToHtml(value, pipeline));
        return true;
    }
}

And that is how my template system works. One final piece to this is to add support for “callouts”. A feature that Obsidian supports natively, and that GitHub has added support for recently. Essentially, I can convert syntax like this:

> [!NOTE] Keep in mind
> Cats are the best animals ever

> [!WARNING] Heads up!
> Just don't buy a snake

Into this:

This system works similarly, parsing the input and translating to HTML, but in a much simpler manner. I'm not going to bother explaining it, but feel free to check out the source.

How do I create new posts?

I've been using Obsidian for a while now, and I'm probably going to continue using it for the foreseeable future. I did initially experiment with developing a WYSIWYG editor for my site, but decided against it because Obsidian does the job just fine. However, having some kind of GUI to upload new posts would certainly be a lifesaver.

Right now, I manually connect to the production database and insert a new record for the post, and paste in the Markdown directly into the row. Do you hate me yet? No? Well you fucking should because that's a very chaotic and dangerous way to work.

At any rate, that's how this blog system was built. Trust me I didn't even cover everything here, there's so much more to this that I haven't even mentioned. But I hope you can now appreciate the absolute level of witchcraft I had to perform to get this to work.