Ministry of SMEs and Startups - Venture Confirmation System Search
Figures
Open full size
Open full sizeOverview
The Venture Confirmation System is the Korean government's register for certified venture businesses — companies apply through it, and the public searches it for scheme guidance, disclosures and support material. I built the search: the indexing path, the REST API, and the guide someone else needed in order to call it. Most of the interesting work on this one was in the plumbing rather than in the retrieval.
The documents were not on the server
The records themselves come out of the database, but their attachments — application forms, guidance documents, the files hanging off notices and FAQ entries — live in S3, not on the machine doing the indexing. That changes the shape of the job. Reading a local file either works or throws; fetching an object over a network can hang, half-succeed, or fail for one file in a batch of hundreds while the rest are fine. So the first thing I wrote was the S3 download method, called per file from the indexing connector, with failures caught per file so one bad object does not take the whole run down with it. Only then does extraction start.
Making a PDF into text
A PDF is not text until something makes it text, and on a government site a lot of the substance is inside the attachments rather than in the page body. Someone searching for a filing deadline should find the form that states it, not just the notice that links to the form. Extraction went through the engine's text filter, backed by Aspose and PDFBox, and the extracted text was indexed alongside the record it belonged to. Results are grouped into three collections: scheme guidance, venture disclosures, and customer support.
The recursive delete, and why it matters
An S3 object key is a path, not a filename — the documents here are keyed by type and upload date, so a key looks like pdf/202103/11/<file>. The download method recreates that structure locally, which means one indexing run does not leave a handful of files behind in a flat folder. It leaves a nested, date-partitioned tree. A delete call will not remove a directory that still has anything in it, so "clean up after yourself" is not one operation. It is a depth-first walk: recurse into each subdirectory, clear its contents, and only then remove the directory itself on the way back up. Get it wrong and nothing fails. The indexing job succeeds, the search works, the tests pass — and every run leaves debris behind until, weeks later, the disk fills up in production.
An API someone else has to call
The search is exposed as a REST API taking keyword, category, sort order, date range and paging; the site's search page is built on it. The deliverable was not just the endpoint but the guide — what each parameter does, what comes back, what happens at the edges — because someone who had not built it had to integrate against it. Writing the interface down is what forces you to notice which parameters only make sense if you already know how the thing works internally.
The query surface
Sort by relevance or by newest, filter by all time / 1 day / 1 week / 1 month / 1 year / custom range, and refine within results — a second query applied to the current result set rather than to the whole corpus, so a user narrows instead of starting over. The period filter earns its place on a site like this rather than being a generic extra: government notices expire, and a two-year-old announcement about an application window is arguably worse than no result at all. Each category also carries its own count, zero counts included, so the shape of the result set is visible before anything is opened.