Are Anti-malware Scanners Causing Spikes in Your MQLs?

Are Anti-malware Scanners Causing Spikes in Your MQLs?

Have you noticed an unusual spike in your MQLs recently? We certainly did.

Fig 1. Marketing qualified leads by week. Note the spike at the beginning of August.

Careful examination of millions of lead activities led us to one explanation: email threat scanners.

We suspect the root cause is anti-malware software vendors adding more stringent checks of email links, probably in response to ransomware attacks like WannaCry. Those changes were likely implemented shortly after the attacks in May 2017, and time frame aligns with the observed increase in email clicks. But the real impact on marketing qualified leads (MQLs) wasn’t until weeks later.

In the hours investigating marketing activities a pattern eventually emerged. A lead would receive an email message and instantly click the contained link. Once it started, the pattern was consistent across all emails sent: an email click would occur within seconds of receipt. We’ll refer to these as auto-click leads.

Suspicious Marketo Auto-click Lead Activities
Fig 2. A lead’s suspicious activities from Marketo. Take careful note of activities order and their timestamps.

Are Your Leads Affected?

All the marketing automation platforms (MAPs) are potentially affected by anti-malware scanners. The situation is industry wide; however, because the issue is email related, first consider if you do the following:

  1. Send frequent emails with a link call-to-action (CTA)?
  2. Score leads clicking those emails?
  3. Allow email clicks to score multiple times?

If none of those apply, then you may not have to worry. But if you do send weekly emails and give a generous score to link clicks, continue reading.

Find Your Auto-Click Leads

Identifying the auto-click leads who follow this pattern isn’t a simple task.

We examined the 51 million activities (so far) in 2017 using a relational database for maximum flexibility. With a full-featured database, we wrote SQL queries to correctly relate every behavior score change to its original email click activity.

Depending on your Marketing Automation Platform (MAP), it may not be possible to identify leads and their specific activities with the same level of precision. A simple cursory search may be sufficient. Try filtering leads to those that have:

  • High lead behavior score (50+ points)
  • Clicked multiple email links (5 or more)

Here is an example smart list setup for Marketo:

Marketo Smart List Criteria
Fig 3. Basic Marketo smart list criteria to find auto-click leads.

Impact on Marketing

Without remediation, what is the impact to your organization?

This influx of MQLs which are not actively interacting with marketing efforts can cause havoc. The scoring system assumes email click activities are a gauge of engagement, but that isn’t the case here. You begin seeing a:

  • decrease in your lead conversion rates to wins
  • decrease confidence in lead quality from the sales team
  • increase in unqualified leads

Update Your Scoring Programs

What actions should you take?

In the short term, disable any global lead scoring programs based on email clicks. Instead measure the outcome of those clicks by scoring click destinations like landing pages. Email scanning software doesn’t execute the JavaScript tracking code; their visits aren’t measured.

Long term solutions require additional effort. Each MAP will require slightly different approaches depending on its features. Also consider how these changes may impact other scoring programs like website visits. Possible solutions:

  1. Independently score each email after a delivery time delay (10–20 minutes)
  2. Add invisible links within emails to detect auto-click leads and decrement
    scores accordingly

Querying for Auto-Click Points

Expand the section below to see the database query used to identify every lead’s auto-click activity and associated score. Running the query yields a list of leads with auto-click scores to help fix lead scores.

 

 

Query Details

We downloaded every activity from our client’s Marketo system into a local PostgreSQL database.Then crafted the following query to get accurate account of leads with number of auto-clicks and associated auto-click behavior points.

WITH lead_clicks AS (
  SELECT DISTINCT ON (score.lead_id, score.id)
    clicks.id            AS click_id,
    sends.id             AS send_id,
    score.id             AS score_id,
    sends.lead_id        AS lead_id,
    sends.activity_date  AS sent,
    clicks.activity_date AS clicked,
    score.activity_date  AS score_scored,
    score.change_value,
    score.new_value,
    score.reason
  FROM activities AS sends
  JOIN activities AS clicks ON (clicks.lead_id = sends.lead_id)
  JOIN activities AS score  ON (score.lead_id = clicks.lead_id)
  WHERE
       sends.activity_date >= '2017-01-01'
  AND clicks.activity_date >= '2017-01-01'
  AND  sends.activity_type_id = 6             -- Send email
  AND clicks.activity_type_id = 11            -- Click email
  AND  score.activity_type_id = 22            -- Change score
  AND  score.primary_attribute_value_id = 918 -- Behavior score
  AND  score.reason ILIKE '%clicks link%'     -- Score due to links clicks
  AND score.id > clicks.id                    -- Only scores after click
  AND clicks.primary_attribute_value_id = sends.primary_attribute_value_id -- With matching program IDs
  AND score.activity_date - clicks.activity_date < INTERVAL '10 minutes'
  AND
    -- Only consider the absolute interval (clicks may be before send)
    CASE 
      WHEN clicks.activity_date - sends.activity_date < INTERVAL '0'
      THEN  sends.activity_date - clicks.activity_date
      ELSE clicks.activity_date - sends.activity_date
    END < INTERVAL '15 minutes'
), lead_click_counts AS (
  SELECT
    lead_id,
    count(*) as email_auto_clicks,
    CASE
      -- Assumes 5 points per click with a max of 100 points
      WHEN count(*) * 5 > 100
      THEN 100
      ELSE count(*) * 5
    END AS auto_click_points
  FROM
    lead_clicks
  GROUP BY lead_id
  HAVING count(*) > 1
  ORDER BY 2 DESC
)
SELECT
  lead_click_counts.lead_id,
  leads.email,
  leads.behavior_score_lead AS lead_behavior_score,
  lead_click_counts.email_auto_clicks,
  lead_click_counts.auto_click_points
FROM lead_click_counts
JOIN leads on (lead_id = leads.id)
;

Executing that query yields a table of leads with associated auto-click behavior points:

Lead Email Behavior Score Auto Clicks Auto Click Points
21886 davel@example.com 80 16 80
31920 rgreen@hq.example.gov 95 16 80
18180 kdemari@example.com 91 15 75
18981 lhall@example.com 80 15 75
10818 isabel@example.com 70 15 75
16182 epatterson@example.com 75 14 70
14572 pamela@example.org 75 14 70
13513 eric@example.com 15 2 10

With this list of leads, we can continue to the next step of correcting their behavior scores.

 

 

Correcting Behavior Scores

How to you fix the behavior scores of auto-click leads?

Conceptually it is a simple task: decrement each lead’s behavior score by the number of auto-click points. But finding and counting the number of auto-clicks from 300,000 leads isn’t possible within the MAPs our clients use. We downloaded the Marketo activities into a local database to get an accurate list of each leads’ auto-click points (see the query section above).

When the auto-click leads are identified, their behavior scores should be decremented to negate all suspicious activity points. In Marketo, you can create separate lists for each group of leads to decrement accordingly.

Resources

Related discussions from the Marketo forums: