Predictive Lead Scoring: Building a Machine Learning Model From Your CRM

Beyond points for page views

Traditional lead scoring is a set of rules someone wrote down: ten points for a demo request, five for a webinar, minus twenty for a student email address. Rules are easy to understand, but they encode guesses, and they rarely get updated when the market changes.

Predictive lead scoring replaces the guesses with a model trained on your own history. It looks at thousands of past leads, learns which characteristics and behaviours preceded a closed deal, and outputs a probability for each new lead. This article walks through how to build one with data most CRMs already hold.

Step 1: define the outcome precisely

A model predicts one thing. Choose it carefully:

• Converted to opportunity within 90 days is a good default. It happens often enough to learn from and arrives quickly.
• Closed-won is closer to revenue but rarer and slower, so you have fewer examples.

Also fix the prediction moment: are you scoring the lead at creation, or after seven days of activity? Everything the model learns from must be known at that moment.

Step 2: build features, and avoid leakage

Features fall into a few groups:

• Fit: company size band, industry, country, job seniority, job function, business vs personal email domain.
• Source: channel, campaign type, landing page, referrer.
• Behaviour: pricing page visits, number of sessions, emails clicked, content consumed, product trial actions, counted over fixed windows like the first 7 days.
• Recency: days since last activity.

The most common mistake is data leakage: using information that only exists because the lead converted. "Number of sales calls" or "deal stage" will make a model look brilliant in testing and useless in production. If a field was filled in by sales after qualification, leave it out.

Step 3: pick a sensible model

Start simple:

• Logistic regression gives a probability and coefficients you can explain to the sales team. Standardise numeric features and one-hot encode categories.
• Gradient-boosted trees (such as XGBoost or LightGBM) usually perform better on tabular CRM data and handle missing values and interactions well.

Train both and compare. If the gain from the complex model is small, the simple one is often the better choice because people will trust it.

Step 4: validate the way you'll use it

Split by time, not at random: train on older leads and test on the most recent months. That mimics reality, where you always predict the future from the past.

Useful metrics:

• AUC: how well the model ranks converters above non-converters overall.
• Lift in the top decile: what share of conversions sits in the top 10% of scores, compared with the average. This is the metric sales feels.
• Calibration: when the model says 20%, do about 20% convert? Check with a calibration plot and fix it with isotonic or Platt scaling if needed.

Conversions are usually rare, so look at precision and recall in the top bands rather than plain accuracy, which can look high even for a model that predicts "no" every time.

Step 5: make it explainable

A score alone doesn't help a rep. Attach the top reasons for each lead, for example using SHAP values: "Visited pricing three times", "Company size in sweet spot", "Came from a partner referral". Reps act on reasons, and the reasons also reveal when the model has learned something odd.

Step 6: deploy and keep it honest

• Score new leads on a schedule or in real time and write the score and reasons back to the CRM.
• Translate probabilities into a few bands (A, B, C) with defined actions for each.
• Monitor drift: if the mix of lead sources or industries changes, performance will drift. Compare predicted and actual conversion each month.
• Retrain on a regular cadence, quarterly is common, and always re-validate before swapping models.

Fairness and data quality

Exclude attributes that could cause unfair or unlawful treatment, and be careful with proxies for them. Watch for feedback loops too: if low-scored leads never get followed up, they never convert, and the model becomes more certain it was right. Reserve a small random slice of low-scored leads for normal follow-up so you keep learning.

Where generative AI adds to this

Classic machine learning handles the scoring. Language models handle the messy text the scoring model can't read: free-text form answers ("we need this before our launch next month"), job titles in dozens of formats, and inbound email content. Turning that text into clean features, like urgency or role category, often lifts model performance more than switching algorithms. A language model can also turn each lead's score reasons into a two-sentence brief for the rep.

For a business like Adasify Retail with a few thousand historical leads, this is a weekend project with open-source tools, not a data-science department.

Key takeaways

• Define one clear outcome and a fixed prediction moment.
• Build fit, source and behaviour features, and ruthlessly remove leakage.
• Compare logistic regression with boosted trees, validated on a time-based split.
• Judge by top-decile lift and calibration, and show reps the reasons behind each score.
• Monitor drift, retrain regularly and keep a random holdout to avoid feedback loops.