Drupal contact / webform submission spike — telling spam from real engagement

1. Problem

Your Drupal site is fine — but your inbox isn't. Since roughly 03:00 UTC, the contact form has been firing one notification email every 4 seconds. The Webform module's "Submissions" tab shows 11,400 entries from the last 6 hours where the previous 6-hour normal was 12. The user-registration form has 340 new accounts with gmail.com addresses that look algorithmically generated ([email protected], [email protected]). Every comment thread is decorated with three-line replies linking to a Russian shoe site.

This is the textbook "drupal contact form submissions spike spam" / "drupal webform getting hammered bots" scenario. It surfaces as a drupal_form_submissions_total rate spike, and the form_type and form_id drilldowns tell you which form the bot found. Honeypot is enabled. CAPTCHA is enabled on /user/register. They are not working — or, more likely, working at 96% effectiveness, and 4% of 50,000 attempts is still 2,000 spam submissions.

The standard advice — "install Honeypot, install CAPTCHA, you're done" — is what got you here. Form-spam detection is a continuous signal problem, not a one-time install.

2. Impact

Form-spam at this volume isn't just noise — it's an active outage of your real lead pipeline. If your sales team triages contact-form messages by hand, hiding 12 real leads inside 11,400 spam ones means roughly 8–10 of those real leads will be deleted along with the spam. Drupal Commerce shops running Webform for B2B quote requests routinely lose six-figure deals this way — the lead gets bulk-deleted on a Friday afternoon and nobody knows.

Operationally, every spam submission writes to webform_submission and webform_submission_data (one row per field). 11,400 submissions × 13 rows = 148,200 rows in 6 hours, plus index updates, plus queued notification emails in queue_mail. Your DB grows, your backups grow, and Drupal cron starts timing out trying to send the email backlog — which surfaces as a separate incident two days later.

For the user-registration form, the cost is sharper: 340 unverified accounts in users_field_data with status = 0 waiting to be promoted by an admin who can't tell the real ones from the synthetic ones. If any get accidentally activated, you've handed authenticated access to a bot.

The trust impact is the quietest: subscribers who actually fill out your form get an auto-reply 11 hours later because the queue is jammed, and never come back. There is no dashboard for "leads that gave up." drupal_form_submissions_total is the only signal that distinguishes the surge from steady-state.

3. Why It’s Hard to Spot

First, the Webform UI counts a successful spam submission and a real submission identically. Both increment the counter, both fire the email handler. Honeypot logs blocked attempts — but a successful spam submission (one that got past Honeypot because the bot waited 5 seconds) is indistinguishable from a real one in dblog.

Second, dblog rotates aggressively. Default is 1,000 rows. A spam burst of 11,400 submissions overwrites your entire watchdog history in the time it takes to make coffee — including the rows that would have shown the attack origin.

Third, there is no out-of-the-box "submissions per form per hour" metric in core. You can see totals on the Webform dashboard and blocked attempts in Honeypot's stats — but the cross-product, successful submissions broken down by form_id over time, requires a custom view, a SQL query, or drupal_form_submissions_total with form_type and form_id drilldowns. Without it you cannot tell whether the surge is your contact form being hammered or a marketing campaign actually working.

Hosting dashboards make it worse: cPanel, Pantheon, and Acquia surface "PHP requests/minute" and "DB queries/second" as health metrics. A spam burst is well-formed POST traffic to a real URL with a 200 response — every dashboard says green.

4. Cause

Drupal's form pipeline routes every form submission — contact, webform, comment, user_register, custom — through \Drupal\Core\Form\FormBuilder::processForm(). When validation passes and the submit handler runs, the Logystera Drupal agent's FormSubmissionSubscriber listens on kernel.terminate and increments drupal_form_submissions_total with two key labels:

• form_type — coarse classification: contact, webform, comment, user_register, custom
• form_id — exact Drupal form ID: contact_message_feedback_form, webform_submission_quote_request_add_form, user_register_form

A second metric, drupal_forms_by_type, aggregates by form_type only — the lower-cardinality companion when you only need "is the surge in webforms or comments?"

Spam bots are not subtle: a script that found /form/contact shows drupal_form_submissions_total{form_id="contact"} rising from 2/hour to 1,800/hour. IP concentration shows in drupal_top_attack_ips; rate-limited submissions appear in drupal_access_denied_total.

Honeypot, CAPTCHA, and Antibot don't prevent the metric from incrementing — they prevent the form save on detection. A successful spam submission is, by construction, indistinguishable from a real one at the form layer. The difference is only visible in rate and IP distribution — which is exactly what drupal_form_submissions_total makes queryable.

5. Solution

5.1 Diagnose (logs first)

The diagnosis flow is: confirm the surge is real, isolate which form_id is being targeted, identify the IP source, then decide between cleanup-and-harden vs. emergency-disable.

1. Confirm the surge in Drupal's submission table directly.

# Webform submissions in the last 6 hours, grouped by form
drush sql:query "
  SELECT webform_id, COUNT(*) AS n
  FROM webform_submission
  WHERE created > UNIX_TIMESTAMP(NOW() - INTERVAL 6 HOUR)
  GROUP BY webform_id
  ORDER BY n DESC;"

A healthy small site sees 0–20 webform submissions per form per day. If you see 1,800 for a single webform_id in 6 hours, you've found the targeted form — which is what drupal_form_submissions_total{form_id="..."} would show as a 100x rate increase.

2. Isolate the IP source from the access log.

# nginx access log: count POST /form/* hits by IP in the last hour
awk -v t="$(date -d '1 hour ago' '+%d/%b/%Y:%H')" \
  '$0 ~ t && $7 ~ /\/form\// && $6 ~ /POST/ {print $1}' \
  /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -20

If the top 3 IPs account for >70% of submissions, you have a single-source bot — easy to block. If submissions are spread across hundreds of IPs (residential proxy network), it's a managed spam-as-a-service operation and IP blocking won't help. This is the same data Logystera surfaces as drupal_top_attack_ips correlated with drupal_form_submissions_total.

3. Inspect submission content for spam fingerprints.

drush sql:query "
  SELECT s.sid, s.remote_addr, d.name, SUBSTRING(d.value, 1, 80) AS val
  FROM webform_submission s JOIN webform_submission_data d ON d.sid = s.sid
  WHERE s.webform_id = 'contact'
    AND s.created > UNIX_TIMESTAMP(NOW() - INTERVAL 1 HOUR)
  ORDER BY s.sid DESC LIMIT 50;"

Spam patterns: URLs in non-URL fields (http:// in a "name" field), Cyrillic or CJK text in an English-only form, identical subject values across different IPs (template reuse), email containing [email protected] patterns.

4. Cross-reference comment spam.

drush sql:query "
  SELECT cid, name, hostname, SUBSTRING(subject, 1, 60) AS subj
  FROM comment_field_data
  WHERE created > UNIX_TIMESTAMP(NOW() - INTERVAL 6 HOUR) AND status = 0
  ORDER BY cid DESC LIMIT 30;"

Unpublished comments (status = 0) sharing a hostname pattern or external domain in the body — that's the bot's signature. Counted in drupal_form_submissions_total{form_type="comment"}.

5. Time-correlate with anti-spam configuration changes.

drupal_form_submissions_total spikes follow a predictable trigger: a recent module disable, a CAPTCHA migration that broke integration, a raised Honeypot threshold, or a new public form launched without protections.

drush config:status | grep -iE "honeypot|captcha|antibot"
drush watchdog:show --severity=warning --count=200 | grep -iE "honeypot|captcha|flood"
# When did the targeted form go live?
drush sql:query "SELECT id, title FROM webform WHERE id='contact';"

If {form_id="contact"} started climbing exactly when Honeypot was disabled or a new webform was published anonymously, that's your story: "spam burst at 03:00 UTC, immediately after webform.contact was published with form_open for anonymous users and Honeypot disabled."

5.2 Root Causes

Every cause maps to a Logystera signal. Prioritized by frequency.

• Honeypot disabled or excluded from the targeted form — the most common cause. Honeypot's protection is per-form opt-in via /admin/config/content/honeypot. A new webform published without the checkbox enabled is fully unprotected. Produces a drupal_form_submissions_total{form_type="webform", form_id=""} step-function increase, with submissions concentrated from 5–20 IPs.

• CAPTCHA module update broke v3 integration — reCAPTCHA v3's silent score check depends on a server-side secret. After a Drupal update or Google API key rotation, validation fails open (passes everything) instead of fail-closed. Produces drupal_form_submissions_total rate increase with no corresponding drupal_access_denied_total entries.

• Anonymous user_register without email verification — Account settings → Visitors with Require e-mail verification off. Bots create thousands of users_field_data rows with status = 0. Produces drupal_form_submissions_total{form_type="user_register"} from many IPs (no concentration), and drupal_top_attack_ips shows long-tail distribution.

• Comment form available to anonymous users — granting post comments to anonymous role is the spam-bot vector that's been in every spam-as-a-service playbook for 15 years. Produces drupal_form_submissions_total{form_type="comment"} with submissions across many entity_id values (every node hit).

• Flood control misconfigured or disabled — Drupal's core.flood service limits failed logins but does not rate-limit successful form submissions by default. If you raised the flood threshold or installed a contrib module that disables it, submissions never trigger drupal_access_denied_total and drupal_form_submissions_total rises unbounded.

• Webform confidential mode masks the real IP — when this is on, remote_addr is NULL'd. Bots submit indefinitely without any IP-based countermeasure detecting them. Produces drupal_form_submissions_total increases with no drupal_top_attack_ips correlation — the spam looks sourceless.

5.3 Fix

Match the fix to the root cause. Don't reflexively turn on every defense — over-protection on real forms causes legitimate users to abandon at 30%+ rates.

Cause A — Honeypot not enabled on targeted form: enable it per-form at /admin/config/content/honeypot (time_limit = 5 seconds). For Webform, also enable Webform's native "Honeypot" element under Settings → Form → Behaviors. The two are independent.

Cause B — reCAPTCHA broken: check drush config:get recaptcha.settings for site_key and secret_key, then drush watchdog:show --type=captcha --count=20. If v3 is failing silently, downgrade to v2 ("I'm not a robot" checkbox) for the duration — fail-closed is better than fail-open.

Cause C — Anonymous registration spam:

drush config:set user.settings verify_mail 1 -y
drush config:set user.settings register visitors_admin_approval -y
drush cr

Then clean up existing spam accounts:

drush sql:query "
  SELECT uid, name, mail FROM users_field_data
  WHERE status = 0 AND created > UNIX_TIMESTAMP(NOW() - INTERVAL 24 HOUR);"
# Review, then bulk-delete:
drush user:cancel --delete-content <uid>

Cause D — Comment spam: revoke post comments from anonymous role.

drush role:perm:remove anonymous "post comments"
drush sql:query "DELETE FROM comment_field_data WHERE status=0
                 AND created > UNIX_TIMESTAMP(NOW() - INTERVAL 6 HOUR);"
drush cr

Cause E — Flood control disabled: restore Drupal core's defaults, plus add a per-IP submission limit via Antibot or a small ECA rule (When Form is submitted → If IP count > 5 in 60s → Deny).

Cause F — Webform purge for the spam burst:

# Purge submissions for ONE form
drush webform:purge contact --all -y
# Or, more surgically, by date:
drush sql:query "DELETE FROM webform_submission
  WHERE webform_id='contact' AND created > UNIX_TIMESTAMP(NOW() - INTERVAL 6 HOUR);"
drush cache:rebuild

If real submissions might be inside the spam window, export first (/admin/structure/webform/manage/contact/results/download), filter, then purge.

5.4 Verify

You're looking for two things to hold simultaneously: drupal_form_submissions_total returns to baseline rate, and the distribution across form_type looks normal again.

# Submissions per form in the last 15 minutes — should match historical baseline
drush sql:query "
  SELECT webform_id, COUNT(*) AS n FROM webform_submission
  WHERE created > UNIX_TIMESTAMP(NOW() - INTERVAL 15 MINUTE)
  GROUP BY webform_id;"

# Honeypot blocks in the last 15 minutes — should be NON-ZERO
# (Honeypot working = it's catching the bots that are still trying)
drush watchdog:show --type=honeypot --count=20

Healthy state in Logystera's entity view: drupal_form_submissions_total{form_type="contact"} at 0–5/hour, {form_type="webform"} matching your historical baseline (varies — 1/hour for a corporate site, 200/hour for a high-traffic publisher), {form_type="user_register"} at 0–3/hour for sites with email verification, {form_type="comment"} at 0–10/hour. The Honeypot-blocked rate should be orders of magnitude higher than the success rate.

The baseline matters: form submissions are not zero in a healthy site — they're steady. Any 10x deviation from your trailing 7-day median for a single form_id over 30 minutes is anomalous. If {form_id="contact"} settles at 3/hour but {form_id="quote_request"} is now firing 90/hour with no campaign live, the bot pivoted to a different form. Whack-a-mole is the operating mode for form spam — the metric tells you which mole popped up next.

If drupal_form_submissions_total reappears above baseline within 6 hours, you addressed one form but the bot pivoted. Go back to 5.1 and find the new form_id.

6. How to Catch This Early

Fixing it is straightforward once you know the cause. The hard part is knowing it happened at all.

This issue surfaces as drupal_form_submissions_total.

Everything you just did manually — query webform_submission, group by webform_id, cross-reference IP origins, time-correlate with the recent Honeypot config change — Logystera does automatically. The Drupal agent's FormSubmissionSubscriber emits drupal_form_submissions_total with form_type and form_id drilldowns on every successful submission, independent of dblog (which would otherwise rotate away the evidence within minutes during a spam burst).

!Logystera dashboard — drupal_form_submissions_total over time drupal_form_submissions_total rate by form_id, last 24h — contact form spike at 03:00 UTC, immediately after Honeypot was disabled on the new webform.

The rule that fires is id 614 — Drupal form submission anomaly, severity warning, threshold rate exceeds 10x trailing 7-day median per form_id over 15 minutes. The 10x trigger is deliberate: small forms (3/day) still alert at 30/day, and large forms (1000/day) don't alarm on noise.

!Logystera alert — Drupal form submission anomaly Warning alert fires within 5 minutes of submission rate exceeding 10x baseline, including form_type, form_id, and the top contributing IPs from drupal_top_attack_ips.

The alert payload includes the timestamp, form_type and form_id (so you know which form to harden before opening Drupal), rate vs. baseline ratio, and top 5 contributing IPs — enough to choose between IP block, Honeypot enable, or emergency form-disable from the alert body alone.

The fix is simple once you know the problem. The hard part is knowing it happened at all. Logystera turns this from a sales-team-deletes-all-leads emergency on a Friday afternoon into a 5-minute notification with the form_id that proves it.

7. Related Silent Failures

• drupal_access_denied_total — Drupal's flood control fires this when a single IP hits the rate-limit threshold. A healthy defense produces both signals: submissions trying, denials catching. The bad shape is submissions rising while denials stay flat.
• drupal_top_attack_ips — IP concentration metric. Single-source bots: top IP > 50% of POST traffic. Spam-as-a-service via residential proxies: top IP < 2% — itself a fingerprint.
• **drupal_http_requests_total{method="POST", path=~"/form/.*"}** — request-layer metric, increments even when validation fails. The ratio against drupal_form_submissions_total is your "validation pass rate"; a sudden ratio change means a defense just broke or got bypassed.
• drupal_forms_by_type — coarse rollup by form_type only. Lower-cardinality companion to drupal_form_submissions_total.
• Email queue backlog from spam notifications — queue_mail fills with notification emails the spam burst generated. Surfaces as drupal_php_error_total two days later when cron times out draining a 50,000-message queue.