Finding a better way
Each time Amsterdam’s residential benefits, a case worker reviews the application for irregularities. If an application looks suspicious, it can be sent to the city’s investigating department – which can lead to a rejection, request to correct paperwork errors, or a candidate may be recommended to receive less money. Investigations can also happen later, once the benefits are dispersed. The result can force recipients to withdraw funds, and even pushes some loans.
Officials have extensive authority over both applicants and current welfare recipients. They can request bank records, ask for beneficiaries in the city hall, and in some cases they make unannounced visits from a person’s home. Since the investigation is conducted-or the paperwork errors may be delayed in the default-maximum payments. And often in more than half of Applications applications, according to data provided by Bedar, the city does not find any evidence of wrongdoing. In these cases, Bodar says, this may mean that the city has “wrongly harassed people.”
The smart check system was finally designed to replace the initial case worker and avoid the scenarios that flags are to send to the Investigation Department. The algorithm will screening applications to identify people who are most likely to include major mistakes based on some personal features, and will redirect these issues to further check through the Enforcement Team.
If everything goes well, the city wrote in its internal documents, the system will improve the performance of its human case workers, flags. Athletes By identifying welfare applicants for investigation A More The proportion of cases with errors. In a document, the city predicted that the model would prevent 125 individual Amsterdamrs from collecting loans and saving 4 2.4 million annually.
There was an interesting possibility for city officials like Smart Check de King, who would manage it when management of the project. He was hopeful, since the city was taking a scientific approach, he says; It will “see if it is going to work” instead of adopting the attitude “it should work, and it doesn’t matter, we will continue it.”
It was such a bold idea that attracted hopeful taxes like Lok Brokers, who is a data scientist who worked on a smart check in his second job just outside college. Speaking at a cafe behind Amsterdam’s City Hall, Brokers remember to be affected by their first contact with the system: “Especially for a project within the municipality,” he says, “It was a very modern project that was trying something new.”
The Smart Check used an algorithm called the “explanatory machine”, which allows people to easily understand how the AI ​​model produces their predictions. Most machine learning models are often considered as “black boxes” that run abstract mathematical processes that are difficult for both employees to use and work for the results.
The smart check model will consider 15 features – which, including the applicant had previously applied for the benefits or received, their assets combination, and the number of their addresses on the file – to assign the risk score to each person. It deliberately avoided population factors, such as gender, nationality, or age, who were thought to have led to prejudice. It also tried to avoid “proxy” factors – such as postal codes – which may not look sensitive to the surface but this may happen, for example, the postal code is associated with a particular ethnic group according to statistics.
In an unusual move, the city has revealed this information and shared several versions of the smart check model with us, and has effectively invited the system’s design and function to check out. With the help of this figure, we were able to make a fictitious welfare recipient to find the insights on how an individual would be evaluated through a smart check.
The model was trained on a data set containing 3,400 former investigations of welfare recipients. The idea was that he would use the results of these investigations conducted by city employees, to find out which factors in the initial requests were linked to a potential fraud.