What do fraudsters want ? And other musings from the RE●WORK AI Fraud Panel

As I had written earlier [Here] we all were looking forward to the panel. Thanks to the distinguished panelists — Mairtin O’Riada/Mousumi Ghosh/Satheesh Ramachandran, the discussions were very illuminating, informative and thought provoking … The credit goes to Katie Pollitt and the RE•WORK team …

Quick notes below …

You should definitely watch the full panel in YouTube it to get the full benefit of the wise words from our panelists …[YouTube Video]

The Panelists:

• Mairtin O’Riada — CTO/Founder of Ravelin, Crime Analysis, Scotland Yard for 10 years — Crime & Criminology
• Mousumi Ghosh, Financial Crime Risk Analytics Leader, Wells Fargo
• Satheesh Ramachandran, Manager, Seller Risk, eBay

As I was transcribing the notes, realized there a ton of words of wisdom and ideas from our distinguished panelists ! Well done, folks !!

The nuances of Fraud & AI

Class imbalance — The class imbalance in fraud (small number of fraud transactions w.r.t non-fraud transactions) makes it interesting. We need to do things like creative sampling techniques, sub-set visualization, selecting appropriate data slices to analyze and so forth, where models in other domains work well without these techniques

Dynamic — A more interesting challenge is the fact that we are against human ingenuity (value judgement aside) and fast changing behaviors. You are being probed for incongruences, holes in your processes, the limits that you set or just blindspots. So we need to pay more attention to things like monitoring Data Drift, Model Degradation and Model Update Cadence

Weak Signals — Tied to dynamic above, any signal from the outside world about fraud is weak at best. Which in turn requires more advanced model architectures and can result in inscrutable models which in turn necessitates explainability

Varied — The fraud domain is varied and each sub-domains have their own interesting twists and nuances — think of fraud models in e-commerce vs AML vs Credit Card vs Identity Theft vs Anomaly Detection.

The Questions :

1. How does AI improve fraud detection — in a broader perspective ?

[Mairtin] Two areas — Promise of Accuracy and the Promise of Basic Automation

• Ability to disconcert interplay between the features and have a much more nuanced view of things; Lifting people out rather than categorizing with a broad brush; for example a customer would put a bounding box on a whole area rather than getting people out of the box
• Frees people who are working on rule and case management to become much more interesting investigators

But you do lose the agency i.e. the ability to pull a lever and make something happen quickly, without thinking about it

[Mousumi] 4 areas, from an AML, Compliance and Sanctions perspective

• Efficiency — Reduction in FP (unproductive alert)
• Effectiveness — increasing the correctly identified suspicious activities (TP)
• Automation — Increasing productivity, freeing up resources
• Advanced Analytics & Results — Accelerate risk scoring, improve alert quality and robust decision making

[Satheesh] Has worked for 20 years

• 10 years ago — claim processing end of the day batch job with all features coming from the claim itself
• Things are radically changed now — to transactional Fraud !
• AI has enabled the high velocity, massive information processing in real time
• Not just structured data coming out of claims file but a confluence of data from different sources — structured, unstructured, image, NLP and so forth
• The signals are much more real time
• Rules are useful because they can respond to things faster; ML models are more broader — so both have their place in fraud detection

2. Do they differ in different verticals ?

• [Satheesh] One big difference is the influence of regulation — to put a model into production takes longer time; in e-commerce, (without regulatory burden) can deploy models quicker
• 2nd, the level of explainability is different at different verticals.
• 3rd, how realtime the system has to be — when the information doesn’t change that much, it can be a batch operation
• [Mairtin] Try to think about fraud from the fraudster’s perspective

What do fraudsters Want ?

A fruitful way is to categorize the fraudsters in your vertical and then use that narrative to build features

• a) The Selfish Fraudster — Want free stuff from somebody else’s money
• b) The Reseller — Interested in Fraud as a business
• c) The Reluctant Fraudster — Uses fraud just as a means to mask their identity because they are in the commission of another crime. e.g. commit fraud intact app not because they want a fee ride but they are in the process committing other crimes like drug dealing and don’t want their movements to be tracked
• d) Larger crime categories — use money laundering to rig the system et al

3. What are the interesting/effective opportunities of AI in your space ?

• [Mousumi]
• a) RPA based solutions
• b) Building advanced AI based models to augment the rules based models
• c) Advanced NLP techniques
• d) Use of platforms — Data Robot is used heavily at WellsFargo
• [Satheesh]
• a) Multimodal features — broad combination of 5 classes of features e.g. structured, graph based embedding, NLP (especially in e-commerce to look at what sellers and buyers are conversing and detect any collusion, or understand the reviews, the exchanges in claims for example), sequential features, image.

Combining all these features to one feature vector in the independent variable space and doing this in an explainable way, is not yet conquered

“What do these embedded vectors mean ?” and “How do they combine with the structured data ?” — is a challenge, even with SHAP, LIME and other explainability algorithms

• b) Fraud is fast changing - how we update the models is another challenge
• c) We learn only what we go after ! Even in claims processing auditors are biases towards what they want to audit. By nature models are built on operational data, which is biased on what you decide to monitor and act on
• [Mairtin] Graphs ! Graph features in live modeling — not only in probabilistic sense but also in cardinality perspective (like fraud rings) and helps to form a narrative

4. What does the future look like ? Transformers in Fraud ? New Algorithms ? New platforms ?

• [Mousumi]
• a) Platforms — they will get more versatile and powerful
• b) New Algorithms
• c) Blockchain
• d) Data being the cloud
• e) Opportunity for people with less ML experience, because of platforms like DR or AutoML with streamlined end to end data pipelines
• [Satheesh]
• Future is not in the algorithms more so much as in the data i.e., features from data. Fraud is a little different from other domains — for example to train a cat vs dog classifier, one can generate data by manipulating the images (like crop, flip, rotate, blurr, denoise and sharpen), in fraud that it not possible
• [Krishna] Extracting weak signals by themselves or combining them with others is an interesting challenge
• [Mairtin] Before thinking about advances in algorithms or feature engineering, we will have to make allowances for what the legislators are demanding. For example, Europe is increasingly prescriptive about how fraud needs to be handled — explicit authentication and prescriptions that fraud should be kept within a certain percentage if authentication is not used

As you can tell, I enjoyed the panel immensely … Hope you do too !