WooAh Choi

Machine Learning Engineer | AI Safety and Evaluation

Machine Learning Engineer with a production engineering foundation, working on AI safety and the evaluation of systems that are harder to verify than they look. MSc AI at Queen Mary University of London, where my dissertation applied mechanistic interpretability to a PPO-trained Transformer policy in partially observable RL. Sole author of a technical report on control boundaries for autonomous research agents. Previously built enterprise search and NLP in production, including the Supreme Court of Korea's e-litigation search.

Get to know me

About Me

MSc AI at Queen Mary University of London, with three years of industry experience as a Software Engineer at Konan Technology building enterprise search and NLP systems. My coursework spans Machine Learning, Neural Networks and NLP, Information Retrieval, Reinforcement Learning, and Conversational Agents — building depth in both theory and implementation.

Most recently I built ResearchAgentLab, an autonomous research agent, and then a control architecture for it after watching it silently substitute a different experiment for the one I had approved. Protocol Before Code freezes approved intent in a machine-checkable protocol, constrains how much the model is allowed to generate, and checks scientific conformance independently of software correctness. In holdout and fault-injection evaluations, 0 of 18 drifted protocols passed and 18 of 18 behaviour-changing trusted-component mutations were blocked before execution; with the content-integrity check neutralised, 15 of those same 18 escaped the remaining stack. A frozen 192-pair oversight experiment found that adversarial curation moved an overseer's target selection from 13.0% to 27.1% while explicit approval stayed at 100% in both conditions. Written up as a sole-authored technical report.

My dissertation is a mechanistic analysis of a PPO-trained causal Transformer policy in a partially observable MiniGrid T-maze. Its central finding was methodological: the aggregate success metric hid a failure mode, because an agent that always turns the same way scores around 50% while having learned nothing about the instruction cue. I designed a worst-case metric, Smin = min(S_key, S_ball), that exposes those one-sided policies, and used causal interventions to test control rather than infer it from correlation. Difference-vector attribution assigned 33–58% of the retrieval difference to padding, yet exchanging the padding source reversed no decision while exchanging the real-transition source reversed both — correlation and causation pointing at different things on the same policy. That signature held in only three of six checkpoints, so equal behaviour did not imply equal mechanism. A parameter-matched LSTM also solved the task at Smin = 0.898, which moved the contribution from architecture comparison to the analysis and evaluation framework itself.

Third author on a short paper accepted to FinNLP 2026 at EMNLP, on whether relevance-ranked web evidence is actually sufficient to verify financial claims. I was one of the two first-pass annotators, labelling 500 of the 750 claim–passage pairs under a blinded four-class protocol, including the 250-pair double-annotated block behind the paper's Cohen's kappa. I also review for the same workshop, the annual workshop of the ACL Special Interest Group on Financial and Economic NLP.

As co-founder of GridFlow Trade (selected for QMUL QIncubator), I built the UK day-ahead price forecasting product on 65,000+ real half-hourly settlement periods. The work I would point at is not the model. Early accuracy looked strong and was wrong: training showed MAE £7.4 while the same model served in production sat at £21.9, because training and inference were not seeing the same features. I rebuilt it as a single point-in-time feature pipeline shared by both paths and verified they produce identical features. The honest figure afterwards is MAE £24.2/MWh against a £34.2 naive baseline, a 29% error reduction on a 90-day held-out window averaged over five seeds. Benchmarking Naive, Ridge, LightGBM, XGBoost and a bidirectional LSTM on one leak-free split showed the tree models and Ridge were indistinguishable within seed noise, so the gains came from feature and evaluation design rather than model choice. I also audited 3.7 years of stored market data against the Elexon API and found a collector date bug that had corrupted 1,634 price and 1,634 system-margin records on every British Summer Time day.

At Konan Technology, I delivered production search and NLP systems for the Supreme Court of Korea, reducing litigation search latency from 5+ minutes to 8 seconds, and shipped query intelligence features across four enterprise clients.

Kaggle: Top ~17% in Google AI4Code · Top ~28% in RSNA 2022 Cervical Spine Fracture Detection.

Machine Learning

PyTorch, transformers and reinforcement learning — from fine-tuning BERT for production NER to training a causal Transformer policy with PPO

Search & Retrieval

Indexing at 100M+ scale, ranking and scoring logic, query understanding and expansion — the retrieval layer the current wave of AI is built on

Evaluation & Interpretability

Worst-case metrics, causal ablations and failure typologies — telling apart systems that work from systems that only score well

Production Engineering

Java, Spring and Python services, data pipelines, Oracle and PostgreSQL at scale, Docker and CI/CD on AWS — models are worth what they are worth in production

Career journey

Work Experience

Konan Technology

Software Engineer, AI Solution Development Team

Nov 2020 — Jun 2023

Built and improved enterprise search and NLP systems across four clients, cutting Supreme Court litigation search latency from 5+ minutes to 8 seconds. Applied BERT-based NER, SFX/SRL query analysis, fastText, and vector search; shipped query intelligence features including related terms, recommendations, popular queries, and typo correction. Alongside the four builds below, I supported 14 shorter engagements across finance, government, healthcare, manufacturing, energy and policing — engine installs and integrations, escalation support, and internal training on the search stack. These included a redundant, high-availability search deployment for the Export-Import Bank of Korea, where the topology the client specified was not achievable under the constraints in place; I prepared the technical case, presented it, and we agreed an alternative that met the availability requirement.

Supreme Court of Korea - E-litigation Search and Analysis

Built a next-generation electronic litigation search and analysis system, reducing search latency from 5+ minutes to 8 seconds and resolving 375+ misanalysed cases. Optimised Java backend algorithms and indexing pipeline to achieve 37× search speed improvement. Applied speech act analysis (SFX) and semantic role labeling (SRL) to derive structured query constraints (time, location, case context). Fine-tuned a BERT-based NER model to extract case numbers, law names, and party names, with 4%+ improvement in extraction quality. Built vector search for related case retrieval and search analytics dashboards (line, bubble, dendrogram charts) for administrators.

Java
Spring
Oracle
Jenkins
GitLab
Python
PyTorch
BERT
Docker
Linux
C++

Millie's Library - Book Search and User Pattern Analysis

Engineered an automated indexing system handling 100M+ data points, designed to scale beyond 1B records. Improved search ranking quality by 38% via custom scoring logic. Implemented auto-completion, related-term suggestion, and typo correction. Delivered time-series analysis of search terms for recommended and popular keyword features.

C++
Python
MySQL
Linux

Heungkuk Fire & Marine Insurance - Main Website Search Renewal

Full-stack development of the web application's search system. Implemented AJAX-based continuous scrolling for smooth user experience and built auto-completion and popular search term features. Supported production rollout with log-based quality monitoring.

Java
JSP
JSTL
JavaScript
AJAX
Oracle
SVN
Linux

Ministry of SMEs and Startups - Venture Confirmation System Search

Developed search engine and RESTful API with text extraction and indexing from documents via Aspose and PDFBox. Implemented recursive file/folder deletion logic for efficient data management.

Java
PostgreSQL
AWS S3
Aspose
PDFBox
REST API
SVN
Linux
What I work with

Skills

Select a skill to see the work behind it. Counts show how many projects and client deliveries match; add more to narrow.

NLP & Search

ML/AI

Data

Languages

Infrastructure

Backend

All work · 16

Konan Technology · Nov 2020 — Jun 2023

Supreme Court of Korea - E-litigation Search and Analysis

Built a next-generation electronic litigation search and analysis system, reducing search latency from 5+ minutes to 8 seconds and resolving 375+ misanalysed cases. Optimised Java backend algorithms and indexing pipeline to achieve 37× search speed improvement. Applied speech act analysis (SFX) and semantic role labeling (SRL) to derive structured query constraints (time, location, case context). Fine-tuned a BERT-based NER model to extract case numbers, law names, and party names, with 4%+ improvement in extraction quality. Built vector search for related case retrieval and search analytics dashboards (line, bubble, dendrogram charts) for administrators.

JavaSpringOracleJenkinsGitLab+20
Konan Technology · Nov 2020 — Jun 2023

Millie's Library - Book Search and User Pattern Analysis

Engineered an automated indexing system handling 100M+ data points, designed to scale beyond 1B records. Improved search ranking quality by 38% via custom scoring logic. Implemented auto-completion, related-term suggestion, and typo correction. Delivered time-series analysis of search terms for recommended and popular keyword features.

C++PythonMySQLLinuxInformation Retrieval+5
Konan Technology · Nov 2020 — Jun 2023

Heungkuk Fire & Marine Insurance - Main Website Search Renewal

Full-stack development of the web application's search system. Implemented AJAX-based continuous scrolling for smooth user experience and built auto-completion and popular search term features. Supported production rollout with log-based quality monitoring.

JavaJavaScriptOracleGitLinux+5
Konan Technology · Nov 2020 — Jun 2023

Ministry of SMEs and Startups - Venture Confirmation System Search

Developed search engine and RESTful API with text extraction and indexing from documents via Aspose and PDFBox. Implemented recursive file/folder deletion logic for efficient data management.

JavaPostgreSQLAWSREST APIGit+4
Project

ResearchAgentLab: Control Boundaries for Autonomous Research Agents

An autonomous research agent, and a control architecture built for it after it silently ran a different experiment from the one that had been approved. Protocol Before Code freezes approved scientific intent in a machine-checkable protocol, binds trusted components by content hash, limits how much the model is allowed to generate, and checks scientific conformance independently of software correctness. Stress-tested with generator-blind holdouts and fault injection.

PythonLLMEvaluation & Benchmarking
Project

Dissertation: Causal Cue Retrieval and Greedy Route Execution — A Mechanistic Analysis of a Transformer Policy

Mechanistic interpretability of a PPO-trained causal Transformer in a partially observable MiniGrid T-maze. Designed a worst-case cue metric that exposed one-sided shortcut policies hidden by aggregate success, then used causal interventions to verify genuine cue-dependent branch control rather than inferring it from correlation.

PyTorchReinforcement Learning (PPO)Mechanistic InterpretabilityPythonEvaluation & Benchmarking+1
Project

Multi-Domain Neural Search Engine — Domain Routing, Hybrid Retrieval and RAG Grounding Evaluation

A six-domain search engine with a domain-specialised BERT encoder behind each domain, a fine-tuned DistilBERT query router, hybrid BM25 and FAISS-HNSW retrieval fused with RRF, and ColBERTv2 late-interaction reranking. A RAG grounding layer was then added on top to answer a question the retrieval metrics cannot: whether a generated answer actually depends on the passages that were retrieved, or on what the model already knew.

PythonBERTCodeBERT / DistilBERTSemantic / Vector SearchInformation Retrieval+11
Project

GridFlow Trade — AI-Powered BESS Trading Platform

A production ML system for UK day-ahead power prices and battery trading, running the whole loop: live ingestion, validation, training, a versioned registry, batch and API serving, drift and service monitoring, automatic rollback or gated retraining, and walk-forward backtesting. The turning point was finding that the reported accuracy had been inflated by target leakage and train/serve skew, and rebuilding it as a single point-in-time feature pipeline shared by training and inference. The honest figure is MAE £24.2/MWh against a £34.2 naive baseline, 29% lower on a 90-day held-out window and 23% lower across a 12-month walk-forward. Public read-only demo on the live site. Selected for QMUL QIncubator.

PythonXGBoostLightGBMPyTorchPostgreSQL+6
Project

Stock AI YouTube Pipeline

Automated weekly AI stock analysis video pipeline using yfinance, OpenAI GPT-4, moviepy, and Stable Diffusion. Fetches market data, generates narrated scripts, synthesises visuals, and uploads to YouTube end-to-end.

PythonLLMData PipelinesGenerative AINLP
Project

BuildU — AI Writing and Application Dashboard

"Build You" — a personal AI dashboard for job applications and English writing. Six tools behind one login: CV feedback with tracked results, an AI-text humaniser, a word-level text diff, IELTS writing practice with band-level scoring, and browser speech-to-text. Next.js 15 on Vercel, GPT-4 for the analysis, Tesseract.js for OCR.

Next.js / ReactLLMWeb Scraping & OCRTypeScriptVercel+3
Project

Invntz Hackathon — E-commerce Aggregator

Web scraping and aggregation platform collecting product data from 5 e-commerce sites via automated browser crawling. Built with Spring Boot, Selenium, and Jsoup. Awarded 2nd place.

SpringWeb Scraping & OCRJavaPostgreSQL
Project

Generative AI Storytelling & Illustration Agent

End-to-end pipeline that converts keyword inputs into narrative stories using LLM APIs, then generates picture-book style illustrations via image-generation APIs. Extended to image-to-video workflows for animated output.

PythonLLMGenerative AIData PipelinesNLP
Project

[ATE] Recipe & Ingredients Management

Full-stack application for managing recipes and tracking kitchen inventory, generating a shopping list from the gap between the two. I designed the data model and built the NestJS/TypeORM backend and its REST API against a Figma prototype. The project stalled when the front-end collaborator left mid-way with the login integration unfinished, so it never reached a usable build. The backend work stands; I intend to finish the client side myself.

NestJSTypeORMPostgreSQLTypeScriptNext.js / React+5
Project

SO Easy — Shared Office Booking Platform

Team project: platform connecting shared office providers with businesses seeking flexible workspaces. My contributions: search engine implementation and kanban board feature. Includes availability, booking management, and payment integration.

NestJSTypeScriptPostgreSQLGraphQLAWS+2
Project

[Kaggle] Google AI4Code — Notebook Cell Ordering

Predicting the next cell to be executed based on the comments and code in Python Jupyter notebooks. Ensemble of CodeBERT and DistilBERT with different weights (0.748 / 0.252) based on the results of running each model.

PythonPyTorchCodeBERT / DistilBERTNLPscikit-learn+4
Project

[Kaggle] RSNA 2022 Cervical Spine Fracture Detection

Predicting the probability of a fracture in each of the seven cervical vertebrae. EDA, translation and summary of the EfficientNetV2 paper, data preprocessing, model training, and inference.

PythonPyTorchComputer Visionscikit-learnpandas / NumPy+2
My work

Featured Projects

Active

ResearchAgentLab: Control Boundaries for Autonomous Research Agents

An autonomous research agent, and a control architecture built for it after it silently ran a different experiment from the one that had been approved. Protocol Before Code freezes approved scientific intent in a machine-checkable protocol, binds trusted components by content hash, limits how much the model is allowed to generate, and checks scientific conformance independently of software correctness. Stress-tested with generator-blind holdouts and fault injection.

Python
FastAPI
SQLite
LLM
Agent Control
Evaluation
Fault Injection
Visit
Research

Dissertation: Causal Cue Retrieval and Greedy Route Execution — A Mechanistic Analysis of a Transformer Policy

Mechanistic interpretability of a PPO-trained causal Transformer in a partially observable MiniGrid T-maze. Designed a worst-case cue metric that exposed one-sided shortcut policies hidden by aggregate success, then used causal interventions to verify genuine cue-dependent branch control rather than inferring it from correlation.

PyTorch
RL (PPO)
Mechanistic Interpretability
MiniGrid
Python
Active

Multi-Domain Neural Search Engine — Domain Routing, Hybrid Retrieval and RAG Grounding Evaluation

A six-domain search engine with a domain-specialised BERT encoder behind each domain, a fine-tuned DistilBERT query router, hybrid BM25 and FAISS-HNSW retrieval fused with RRF, and ColBERTv2 late-interaction reranking. A RAG grounding layer was then added on top to answer a question the retrieval metrics cannot: whether a generated answer actually depends on the passages that were retrieved, or on what the model already knew.

Python
BERT
DistilBERT
FAISS
BM25
ColBERTv2
RAG
LLM
Statistical Testing
Active

GridFlow Trade — AI-Powered BESS Trading Platform

A production ML system for UK day-ahead power prices and battery trading, running the whole loop: live ingestion, validation, training, a versioned registry, batch and API serving, drift and service monitoring, automatic rollback or gated retraining, and walk-forward backtesting. The turning point was finding that the reported accuracy had been inflated by target leakage and train/serve skew, and rebuilding it as a single point-in-time feature pipeline shared by training and inference. The honest figure is MAE £24.2/MWh against a £34.2 naive baseline, 29% lower on a 90-day held-out window and 23% lower across a 12-month walk-forward. Public read-only demo on the live site. Selected for QMUL QIncubator.

Python
XGBoost
LightGBM
PyTorch
MLflow
SciPy HiGHS
FastAPI
PostgreSQL
Docker
GitHub Actions
Elexon API
NESO API
Visit
Completed

Stock AI YouTube Pipeline

Automated weekly AI stock analysis video pipeline using yfinance, OpenAI GPT-4, moviepy, and Stable Diffusion. Fetches market data, generates narrated scripts, synthesises visuals, and uploads to YouTube end-to-end.

Python
OpenAI GPT-4
yfinance
moviepy
Stable Diffusion
Completed

BuildU — AI Writing and Application Dashboard

"Build You" — a personal AI dashboard for job applications and English writing. Six tools behind one login: CV feedback with tracked results, an AI-text humaniser, a word-level text diff, IELTS writing practice with band-level scoring, and browser speech-to-text. Next.js 15 on Vercel, GPT-4 for the analysis, Tesseract.js for OCR.

Next.js 15
OpenAI GPT-4
Tesseract.js
TypeScript
Vercel
Visit
Completed

Invntz Hackathon — E-commerce Aggregator

Web scraping and aggregation platform collecting product data from 5 e-commerce sites via automated browser crawling. Built with Spring Boot, Selenium, and Jsoup. Awarded 2nd place.

Spring Boot
Selenium
Jsoup
Java
PostgreSQL
Prototype

Generative AI Storytelling & Illustration Agent

End-to-end pipeline that converts keyword inputs into narrative stories using LLM APIs, then generates picture-book style illustrations via image-generation APIs. Extended to image-to-video workflows for animated output.

Python
OpenAI
Stable Diffusion
LLM APIs
Pipeline Design
Paused

[ATE] Recipe & Ingredients Management

Full-stack application for managing recipes and tracking kitchen inventory, generating a shopping list from the gap between the two. I designed the data model and built the NestJS/TypeORM backend and its REST API against a Figma prototype. The project stalled when the front-end collaborator left mid-way with the login integration unfinished, so it never reached a usable build. The backend work stands; I intend to finish the client side myself.

NestJS
TypeORM
PostgreSQL
TypeScript
React Native
REST API
Code
Completed

SO Easy — Shared Office Booking Platform

Team project: platform connecting shared office providers with businesses seeking flexible workspaces. My contributions: search engine implementation and kanban board feature. Includes availability, booking management, and payment integration.

NestJS
TypeScript
PostgreSQL
GraphQL
AWS
Code
Completed

[Kaggle] Google AI4Code — Notebook Cell Ordering

Predicting the next cell to be executed based on the comments and code in Python Jupyter notebooks. Ensemble of CodeBERT and DistilBERT with different weights (0.748 / 0.252) based on the results of running each model.

Python
PyTorch
CodeBERT
DistilBERT
NLP
scikit-learn
Completed

[Kaggle] RSNA 2022 Cervical Spine Fracture Detection

Predicting the probability of a fracture in each of the seven cervical vertebrae. EDA, translation and summary of the EfficientNetV2 paper, data preprocessing, model training, and inference.

Python
PyTorch
EfficientNetV2
Computer Vision
scikit-learn