Implementing AI Effectively in Limited Public Sector Settings

SLMs are tailored specifically for the needs of the department or agency utilizing them. Data is securely stored outside the model and is accessed only upon query. Carefully designed prompts ensure that only the most relevant information is retrieved, leading to more accurate responses. By employing methods such as smart retrieval, vector search, and verifiable source grounding, AI systems can be developed to meet public sector requirements.

Therefore, the next stage of AI adoption in the public sector may involve bringing the AI tool closer to the data, rather than transferring the data to the cloud. Research suggests that by 2027, small, specialized AI models will be utilized three times more than large language models (LLMs).

Superior search capabilities

“When people in the public sector hear AI, they probably think about chatbots. But there’s much more potential,” says Xiao. “AI can transform how the government searches for and manages the extensive data they possess.”

Looking beyond chatbots unveils one of AI’s most immediate opportunities: significantly enhanced search functionalities. Like many organizations, the public sector contends with vast amounts of unstructured data—including technical reports, procurement documents, minutes, and invoices. Modern AI can deliver results derived from various media forms, including readable PDFs, scans, images, spreadsheets, and recordings, in multiple languages. All of this can be indexed by SLM-powered systems to provide personalized responses and to draft complex texts in any language while ensuring outputs comply with legal standards. “The public sector holds a wealth of data, but they often lack the knowledge to utilize this data effectively. They may not be aware of its full potential,” says Xiao.

Even more significantly, AI can assist government employees in interpreting the data at their disposal. “Today’s AI can offer a fresh perspective on harnessing that data,” notes Xiao. A well-trained SLM can interpret legal norms, extract insights from public consultations, support data-driven executive decision-making, and enhance public access to services and administrative information. This can lead to substantial improvements in how the public sector operates.

The small-language promise

Shifting the focus to SLMs changes the discussion from the comprehensiveness of the model to its efficiency. LLMs require significant performance and computational resources and specialized hardware, which many public entities may find prohibitive. Although some initial investment is necessary, SLMs are generally less resource-intensive than LLMs, making them more cost-effective and environmentally friendly.