Targetting the Low-Hanging Fruit

Non-refoulement judgments are highly standardised with few moving parts, which make them ideal targets for automation with LLM. We limit our scope to typical non-refoulement cases (no children, no multiple applicants and no merit). Instead of spending time and money on a comprehensive system that works on all non-refoulement cases, we quickly push out a system that targets the lowest hanging fruit ripe for LLM assistance. If we adopt a Microservice design pattern, we can re-use some of the computer code in the next iteration of the program and build upon the lessons learnt in this small system.

Why traditional technique disappoints

Many LLM courses teach us that, in order to generate a special form of writing, we should write long prompts filled with guidance and examples. This technique can work, but their success rates vary. In most cases, the LLM model needs to be complex and a lot of testing and fine-tuning is needed. Regardless of the effort and time put in, this technique can be quite temperamental and there are many cases where the LLM model abandons the given examples completely, despite the best effort to remedy it. This phenomenon is often called "prompt-brittleness" and is a source of headache for LLM developers across the world, see Commey, D. (2026), When “Better” Prompts Hurt: Evaluation-Driven Iteration for LLM Applications. Avoiding the common pitfalls with hydration pattern

Because non-refoulement cases have a highly predictable and rigid structure, we can avoid prompt-brittelness altogether with the Hydration Pattern, see Frank J.E. Flitton, “A Practical Pattern for Hydrating AI-Generated Object Templates”.

With a Hydration Pattern, we first create a traditional computer program to write the mundane portion of the CA judgment. We then rely on LLM to extract the information from the case paper. The information from the LLM to then used to "hydrate" (or bring-alive) the traditional computer program that generates the CA judgment. In the demonstration below, black text represents the immovable text from the traditional computer program while the green text represents the information supplied by the LLM.

 
Traditional Computer Program
CACVCase Number / Case Year - Applicant Name. This is the Applicant’s appeal against the decision of CFI Judge Title Judge Name (“the Judge”) given on Dismissal Date refusing his / her / their application for leave to apply for judicial review (“Leave Application”) against the decision of the Torture Claims Appeal Board (“the Board”) dated Board Date(s).

The files below show that the concept works: a judgment for CACV 374/2026 was generated programmatically from two scanned PDFs retrieved from iCMS. The input PDFs, generated result, and redacted Python code are enclosed for further action.

The computer code is purely for demonstrating LLM's capability and design pattern only. The author of this report is not a professional software engineer and significant effort is needed to turn the crude computer code into a reliable computer system. Careful consideration is needed for security, cost monitoring, performance enhancement and handling feedback from users.