SLMs are tailored to the needs of the department or agency that will use them. Data is stored securely outside the model, and is only accessed when a query is made. Carefully designed prompts ensure that only the most relevant information is retrieved, providing more accurate answers. Using methods such as Clever recovery, Vector searchand Grounding by verifiable meansAI systems can be built that meet the needs of the public sector.
Thus, the next step in the adoption of AI in the public sector may be to bring the AI ​​tool to the data rather than sending it to the cloud. Gartner predicted. That by 2027, small, specialized AI models will be used three times more than LLMs.
Advanced search capabilities.
“When people in the public sector hear AI, they probably think of ChatGPT. But we can get a lot more excited,” says Xiao. “AI could revolutionize how government explores and manages the vast amounts of data it holds.”
Looking beyond chatbots reveals one of AI’s most immediate opportunities: dramatically improved search. Like many organizations, the public sector has mountains of unstructured data—including technical reports, procurement documents, minutes and invoices. However, today’s AI can deliver results from mixed media, such as readable PDFs, scans, images, spreadsheets, and recordings, and in multiple languages. All of these can be configured through SLM-enabled systems to provide tailored responses and draft complex texts in any language, while ensuring that outputs are legally compliant. “The public sector has a lot of data, and they don’t always know how to use that data. They don’t know what the possibilities are,” says Xiao.
Even more powerfully, AI can help public servants interpret the data they access. “Today’s AI can give you a whole new perspective on how to use that data,” Xiao says. A well-trained SLM can interpret legal principles, extract insights from public consultations, support data-driven executive decision-making, and improve public access to services and management information. This can dramatically improve the way the public sector works.
The promise of a small language
Focusing on SLMs shifts the conversation from how comprehensive a model can be to how effective it is. LLMs have significant performance and computational costs and require specialized hardware that many public institutions cannot afford. Despite requiring some capital expenditure, SLMs are less resource intensive than LLMs, so they are cheaper and have a reduced environmental impact.