IUU+DB: Tracking Illegal, Unreported, and Unregulated Fishing, Seafood Fraud, and Labor Abuse
through LLM-driven Information Extraction

IUU+DB is an end-to-end pipeline for turning unstructured documents into structured incident records. It ingests documents, classifies them by source type, and extracts relevant information into a standard output schema. The pipeline is guided by an input schema that defines the document categories to recognize and the key data elements (KDEs) to extract. Fig. 1 gives an overview of the IUU+DB pipeline. Based on the KDEs, IUU+ types, and behaviors identified by our team outlined in Appendix A, we classify each source into one of three main source types: (i) sources that describe one or multiple incidents, (ii) sources that are related to IUU+ fishing but do not pertain to an incident, and (iii) sources unrelated to IUU+ fishing. For each incident, the system extracts information from 14 KDE groups covering 100 possible fields, classified by 7 possible IUU+ types, and 41 behaviors. For sources classified as related to IUU+ but that do not describe a specific incident, the system extracts a set of 3 KDEs. To manage the large extraction task possible, the pipeline first aims to ascertain which KDE groups are present in a document and then extracts fields from only those groups. This effectively reduces the schema information the LLM needs to hold in context at any given moment.

Document Pre-Processing Module is the first step of IUU+DB and accepts plain text, URLs, and PDFs. For PDFs we use Pytesseract ¹¹1https://pypi.org/project/pytesseract/ to perform Optical Character Recognition (OCR) to render documents ingested into plain text. For URLs we first attempt to extract text data with Playwright ²²2https://playwright.dev/python/; if that fails, we use an LLM to extract the core text. Once the text is extracted, the system removes formatting artifacts, checks for duplicate documents using a text hash, and skips documents that have already been processed. Finally, IUU+DB splits the cleaned text into smaller chunks, embeds them using Qwen3-VL-Embedding-2B ³³3https://huggingface.co/Qwen/Qwen3-VL-Embedding-2B, and stores them in a vector database for later semantic search. We chose this Qwen model as it supports a token length of up to 32k which is more than enough and supports input that includes a mixture of text and image data, so in the future we can incorporate embedded figures from sources. This step can be parallelized so that large document collections can be processed efficiently.

The Source Classification Module determines what kind of information a document contains. Given a document and a set of possible source types, the module first identifies passages that appear relevant to IUU+ activity.

Our framework uses few shot prompting to classify the document as one of the following: a source describing an IUU+ incident, a source related to IUU+ fishing but with either more general information or not enough sufficient information provided to be considered a specific incident, or a source unrelated to IUU+ fishing. Because a single document may describe more than one incident, this module also extracts and stores the passages associated with each incident. This helps the KDE extraction module process each incident separately and reduces confusion in later stages.

The KDE Extraction Module is the workhorse of the IUU+DB system. It takes the source document, scope classification, as well as optionally relevant passages and a list of KDE groups. The extractor then makes runs where is the number of relevant passages, or if the classifier did not include them. In each run the extractor first determines the presence of each KDE group given before running the extraction of each KDE fields within the group. If the KDE groups were not given the module defaults to extracting every field. For our implementation we again utilized an LLM with DSPy⁴⁴4https://dspy.ai/. DSPy helps us define a structured schema for extraction and encourages logical consistency across fields. For example, if the system identifies a seafood product, we expect it to also identify the species from which that product is made. From the three source classification, we identify two schema to extract: one for incidents, and another albeit much smaller schema for related to IUU+ fishing which tracks species, countries, and companies mentioned. We do not capture any additional information from unrelated articles. In our experience, we found that some KDEs are relatively rare, and unlikely to occur together within the same incident (e.g. It would be uncommon for there to be a single IUU+ incident as we have described them, to include data about both catch AND aquaculture.) We grouped KDEs therefore by those most likely to co occur with one another and extract them in the modules runs. After extracting the KDEs we then classify the IUU+ behavior exhibited in the incident.

After extraction, the incidents are aggregated in two ways, first by merging duplicate incidents and secondly by grouping related incidents into trends. The De-Duplication and Trend Identification modules first identifies similar incidents using cosign similarity against their sources, we then check whether the duplicate candidates have matching identification information such as vessel name, or id and matching event or enforcement dates. If the two incidents pass those filters and have high cosign similarity, the module merges the extracted KDEs. In the cases of conflicts, the module queries all attached sources for the relevant KDE field in order to get a consensus.

Using DSPy’s prompt optimizers and manually validated data we automate the iterative task of prompt fine tuning. we utilized MIPROv2  (Opsahl-Ong et al., 2024) to iteratively finetune the prompt and field descriptions themselves. For our metric we assigned rewards for having a correctly filled KDE field with high cosine similarity between the extraction and the validated field. To penalize hallucinations we negatively reinforce incorrect extractions.

We collected data for this project in two primary ways, directly from APIs and from websites via webscrapers. We used the following APIs: NewsAPI (NewsAPI.ai, 2026), SerpAPI (SerpAPI.com, 2025), and CORE API (Knoth et al., 2023). And also targeted the following websites via webscrapers: MongaBay (MongaBay, 2026), US DOJ (US DOJ, 2026), Oceana (Oceana, 2026), US NOAA Fisheries (NOAA Fisheries, 2026), and Undercurrent News (Undercurrent News, 2026a). To create queries for candidate source discovery, we used the previously defined IUU+ types and behaviors to build queries that would capture the wide range of behaviors (see Appendix A).

For webscraping, we separated websites into two groups by how specific the sites were to the fish-seafood sector. For more general sites, e,g., MongaBay and DOJ, we searched for the following key words: ”illegal fishing”, ”IUU fishing”, ”overfishing”, ”fishing violation”, ”illegal catch”, ”seafood fraud”, ”forced labor fishing”, ”illegal aquaculture”, and ”illegal seafood sanctions”. In all these searches, our aim was to capture any relevant articles capturing the illicit behavior as well as narrow the scope to only fishing and associated trade. Since the second group of sites focus solely on the fish-seafood sector (e.g., nongovernmental organization (NGO) websites, NOAA Fisheries, and Undercurrent News), it was the case that if the article referred to an infraction it was highly likely to be about IUU+ behaviors. As such, we selected the following keywords: ”violence”, ”investigation”, ”coercion”, ”arrest”, ”charge”, ”indict”, ”fined”, ”enforce”.

With the three APIs we can use more complex logic and selected the following query: ”IUU or seafood mislabeling or ((Transhipment or sanctions or unfree labor or workplace violations or wage theft) and (ship or seafood or fish) and (arrest or criminal investigation or indictment or search or seizure or fine))”. From NewsAPI, instead of using keywords, the query was made up of ”concepts” which utilize a semantic searching mechanism allowing us to pull articles from different languages, and related articles that may not use the exact key words.

In addition to exposing APIs for general use and further analysis, we developed a web interface to help subject-matter experts analyze, edit, and audit the data. The interface has five main pages. The Insights page shows statistics and figures from the database. The Sources page allows users to read and explore the documents that have been ingested. The Incidents and Related to IUU+ pages allow users to review the KDEs extracted under each schema. The Upload page allows administrative users to add new documents to the database from plain text, URLs, or PDFs. The Sources, Incidents, and Related to IUU+ pages also include version history, so administrators can track changes made to each record (see fig 2).

We find that the data set and IUU+DB is capturing a wide scope of incidents, from high profile cases (AFP, 2022; Undercurrent News, 2025), to regional conflicts (Firstpost.com, 2023; Island.lk, 2023; Undercurrent News, 2024), to local news (Patrick Gouldsbrough, 2022; Environment Agency of the United Kingdom, 2024).

From Fig. 3 we can see that while English speaking nations are highly represented, the dataset has captured events from every continent and nearly every coastal state. The two largest trends we extracted were Illegal Red Snapper harvesting in the Gulf of Mexico and Sri Lankan-Indian fishing rights conflicts.

For a given incident, IUU+DB extracts roughly 20% of the fields in the full schema; see Fig. 4. This is not surprising given the size of the schema and the kinds of articles that make up most of the database. We next asked whether IUU+DB fills the fields we would expect for each IUU+ classification. For this purpose, we first analyzed the KDE Presence detection module, which decides whether the system should search for fields within a given KDE group. We then measured KDE group presence across IUU+ types. Figures 4 and 5 show that some KDE groups, such as event and compliance information, appear across many IUU+ types. However, behavior-specific KDEs are much more common in the categories where they are expected. For example, labor standards and crew information increase from 7% and 16% across all cases to 81% and 50% in labor abuse cases. Similarly, aquaculture information increases from 3% across all cases to 70% in aquaculture-related cases.

The second analysis we used biclustering to find coordinated occurrences of KDEs and IUU+ sub behaviors. We use the FP-Max algorithm ¹¹1https://rasbt.github.io/mlxtend/user_guide/frequent_patterns/fpmax/ wherein each transaction is modeled as a KDE with one or more IUU+ subtypes. To account for data imbalance, we ran the biclustering in two stages. The first stage was conducted with a minimum support threshold of 0.25 (which yields primarily transactions of IUU+ type of ”Illegal Fishing”), and the second with a minimum support threshold of 0.1 after filtering out transactions from the first step. The maximum set bicluster we found was made up of the following 10 KDEs all from the Event group: event and enforcement country, event and enforcement location, and event and enforcement location category, enforcement category, primary offender, event date, and resolution. This cluster was supported by 45% of transactions and by 36 unique IUU+ sub-behaviors.

In the second run, there were four clusters each with 10 KDEs, with the most common being the made up of the same 10 KDEs from the first run. The other three were the same as the set above, less event date and with a different KDE instead that is highly relevant to a specific IUU+ behavior or type. Such as, species or product information for Species Fraud and Mislabeling, or crew information information in cases of Labor Abuse.

We aimed to determine the most surprising illegal fishing incident that occurred within our dataset. For this purpose, we employed a maximum entropy approach. We described each event in terms of an (enforcement country, Illegal fishing sub behavior) tuple. We learn a maximum entropy distribution using iterative proportional fitting (IPF). For each quarter we use incidents from the past year (i.e., past four quarters) to populate the seed matrix which IFP uses to estimate the current quarter. For each cell in the country/sub behavior matrix we then calculated the difference between the estimated and observed, and render it as a z-score. Using to define a surprising event, we discover 46 surprising country-behavior events, the top 6 seen in fig 2. Examples of these identified surprising incidents are the USA Q2 2022, and LKA 2024 Q3, the first of which involved two Florida men fishing prior to the season starting and catching undersized fish  (Undercurrent News, 2022), and the second was a case of falsifying tracking data in Saya de Malha, northeast of Madagascar  (Ian Urbina, 2024).

For our evaluation set, we had three validators manually label a random selection of 103 source documents, 52 of which were initially classified by IUU+DB as incidents, 25 were related to IUU+ but not involving an incident, and 26 unrelated to IUU+. The validators read through the document and determined the source scope. If the validator found the document to be an incident, they proceeded to classify the incident from the 7 IUU+ types and 41 sub behaviors. Finally the validators extracted the 100 KDEs and enter them into IUU+DB. We log any differences and update the database with the human labeled data.

We first evaluate classification performance on the manually validated test set. We report precision, recall, and F1 score for document scope, IUU+ type, and IUU+ sub-behavior. We use the same metrics to evaluate extraction performance at the field level, and then aggregate the results by KDE group. We also report a mismatch rate. This measures how often the system’s extracted value differs from the validated value when both contain a populated field.

The classification results in Table 3 show that IUU+DB performs reasonably well across the main classification tasks. However, the confusion matrix, seen in fig 6 and the document-scope results show some weaknesses. For example, incident classification has a precision of 0.50 and a recall of 1.00. This suggests that the classifier is tuned to avoid missing incidents, but it also marks too many documents as incidents. This low precision adds noise to the database and makes downstream analysis more difficult.

For sources classified as incidents, we evaluated extraction performance; the results are shown in Table 4. We observe four main patterns: high recall with high mismatch, moderate-to-high recall with low mismatch, low recall with low mismatch, and low-to-moderate recall and high mismatch. In the high-recall, high-mismatch groups, such as species and products, the system often finds the relevant information but also includes noisy or incorrect values. In the moderate-to-high-recall, low-mismatch groups, such as compliance and vessel information, the system performs well and extracts accurate information at a reasonable rate. In the low-recall, low-mismatch groups, such as aggregation, distribution, and landing, the system often misses relevant fields. However, when it does extract them, the values are usually correct. Finally the last group are the fields that the system does not often extract, and when it does it often struggles to do so with high degrees of accuracy, such as information regarding catch. or labor standards. These KDE groups highlight areas for improvement with more training examples for prompt optimization.

From these statistics we can see that while IUU+DB generally performs well, there is more work needed to reduce the amount of noise, by improving the precision of the scope classifier and reducing the mismatch rate.

To evaluate whether IUU+DB improves performance we compare the results of the test set above against two base lines: GPT-4o Mini  (OpenAI et al., 2024), the model used in our implementation, as well as GPT 5.4 Mini (OpenAI et al., 2026) to test how IUU+DB performs w.r.t. newer models.

As can be seen from Table 5, all three models perform well. IUU+DB offers a slight improvement on the initial task of classifying the scope against both stand alone LLM calls, but shows 15-20% improvement over the plain GPT-4o calls and  10% improvement over the newer GPT-5.4 model for IUU+ Type and Sub-Behavior respectively. Additionally, we noticed that both GPT-4o Mini and GPT-5.4 Mini extracted about 5 fields fewer than the human extraction or than IUU+DB did; see Fig. 7.

Future work will focus on improving extraction accuracy and strengthening source-scope and IUU+ type classification. Better classification will reduce noise in the database and make it easier for subject matter experts to identify useful patterns and insights. We will also use the generated labels, along with the manually labeled fine-tuning set, to develop and evaluate alternative classification modules that can improve IUU+DB’s performance.

We also plan to expand the system to ingest image and figure data. Some important information may appear only in graphics, tables, maps, or other visual formats, and adding this capability would allow IUU+DB to capture a broader range of evidence from each source.

Another area of future work is the development of a more user-friendly record validation pipeline that can be opened up to external users. As the database grows, expert validation will remain important for improving record quality, correcting errors, and helping the system learn from reviewed examples.

Finally, we plan to move IUU+DB toward a regularly updated database that can be refreshed on a weekly basis. This will include improved extraction pipelines for each source type and broader coverage of web-scraped sources, including additional NGO, government, industry, and media sources from around the world.