How to Automate Your Recon with Subfinder, httpx, and Nuclei
October 02, 2026

How to Automate Your Recon with Subfinder, httpx, and Nuclei

Recon is the least glamorous part of bug bounty and usually the part that pays. Finding a subdomain nobody has scanned yet remains the most reliable way to land a valid report, and it is work you can hand to a script. Subfinder, httpx and Nuclei chain together cleanly: one finds names, one finds which of those names are alive, one asks the live hosts a few thousand questions. Wire them into a pipeline and recon stops eating your afternoons.

The setup below is deliberately boring. One script, three stages, timestamped output you can compare with yesterday's run. Nothing here needs a framework or a paid service.

The Three Stages of a Recon Pipeline

Think of recon as a funnel. Each tool narrows the list and adds context, and each stage should be able to run without the next one.

Subfinder: names from public sources

Subfinder queries certificate transparency logs, search engines, DNS datasets and other passive sources, then prints the subdomains it found. It never touches the target directly, which makes it the safest place to start. Add API keys for the sources that offer them — the provider config lists the options — and your result set grows noticeably. Without keys it still works, just with fewer sources behind it.

httpx: which names actually answer

A list of 4,000 subdomains is not a list of 4,000 targets. Most are dead, parked, or resolve to the same load balancer. httpx takes that list, requests each host, and returns the ones that respond along with status codes, page titles, content length and detected technologies. This is where a scan becomes usable: a host returning 200 with the title "Jenkins" is a different conversation from one returning 403.

Nuclei: asking the live hosts questions

Nuclei runs community-maintained templates against your live hosts — exposed config files, misconfigured headers, known CVEs, default-login hints. It is fast, but it is not subtle. Point every template at a thousand hosts and you generate noise for the target, your connection, and your own triage queue.

Install Once, Then Leave It Alone

All three tools are written in Go. With a working Go environment, install them the same way and keep them updated — signatures and templates move constantly, and a stale Nuclei is a quiet source of false negatives.

go install -v github.com/projectdiscovery/subfinder/v2/cmd/subfinder@latest
go install -v github.com/projectdiscovery/httpx/cmd/httpx@latest
go install -v github.com/projectdiscovery/nuclei/v3/cmd/nuclei@latest
nuclei -update-templates

Each binary also supports -up to update itself and -duc to skip the update check when you want quiet cron output. For Subfinder, put your API keys in ~/.config/subfinder/provider-config.yaml and test them against a domain you own before you trust the results.

A Pipeline in One Bash Script

Save this as recon.sh, pass a domain as the first argument, and it writes a dated folder per target.

#!/usr/bin/env bash
set -euo pipefail

DOMAIN="$1"
OUT="recon/$DOMAIN/$(date +%F)"
mkdir -p "$OUT"

subfinder -d "$DOMAIN" -all -silent -o "$OUT/subs.txt"
httpx -l "$OUT/subs.txt" -silent -sc -title -td -o "$OUT/live.txt"
awk '{print $1}' "$OUT/live.txt" | nuclei -severity low,medium,high,critical -rl 150 -o "$OUT/nuclei.txt"

A few details matter. set -euo pipefail stops the script instead of letting it leave half-finished output behind. -all tells Subfinder to use every source rather than the fast subset. -sc -title -td gives httpx enough context for triage later. The awk step strips that context back off, because Nuclei expects plain URLs. And -rl 150 caps requests per second, which keeps you a decent guest on someone else's infrastructure.

Tuning Out the Noise

The first run is always ugly. Adjust these before pointing it at every program you follow:

  • Filter wildcards early. If every subdomain returns identical content length, use httpx's -fl and -fc filters to drop those responses instead of triaging them by hand.
  • Narrow Nuclei by tag. Start with -tags cve,exposure,misconfig on a new target and widen only when you have time to read the results.
  • Exclude the destructive stuff. -etags dos,fuzz keeps denial-of-service and fuzzing templates out of automated runs. Nothing automated should be stress-testing a third-party service.
  • Keep the intermediate files. They are cheap, and they let you re-run a later stage without repeating the earlier one.
  • Reuse the diff. Because each run gets its own dated folder, you can point Nuclei at yesterday's new hosts alone instead of rescanning everything.

Sorting what deserves attention

After a few runs you will notice the same three things producing findings: staging and development hosts, admin panels on non-standard ports, and old services someone forgot about. Add -p 80,443,8080,8443 to httpx if the program allows it, then grep your titles for words like "staging", "test", "admin" and "jenkins" before you read the rest.

Scheduling and Diffing

Recon gets interesting on the second run. A daily cron job plus a diff turns a one-off scan into monitoring:

0 6 * * * /home/you/recon.sh example.com >> /home/you/recon.log 2>&1

Then compare two days of subdomain lists with comm. Sort both files first, or the output is nonsense:

sort recon/example.com/2025-05-01/subs.txt > /tmp/a
sort recon/example.com/2025-05-02/subs.txt > /tmp/b
comm -13 /tmp/a /tmp/b

What comes back is the handful of subdomains that appeared since yesterday — often hosts someone stood up quickly with no authentication in front of them. That list is short enough to check by hand, which is exactly what you want from automation.

Stay Inside the Lines

Automated scanning is not a licence to hit everything that resolves. Read the program's scope, exclude out-of-scope assets explicitly — a deny list in your script beats a good memory — keep rate limits low, and log what you ran and when. If a program forbids automated scanning of certain hosts, respect that. When the scope wording is ambiguous, ask the program owner or take legal advice before you scan; a mistake there costs far more than any bounty.

Run It, Then Read It

The point of this pipeline is not to take you out of recon. It is to take out the typing so your attention lands on the interesting part: the odd page title, the 500 on an admin path, the staging host with a login form and no rate limiting. Build the script once, run it on a schedule, and spend your mornings reading diffs instead of retyping flags.

Photo: dmncwndrlch / Pixabay

Previous Post

From Zero to P1: A Writeup of a Critical Authentication Bypass

Next Post

Transitioning from IT Security to Full-Time Bug Bounty: A Realistic Roadmap

Related Posts

About Us

Bug Hunter Journal

Bug Bounty

Notes from the front lines of bug hunting.

Read More

Latest Post

Understanding CVSS Scoring: How to Rate Your Bugs Like a Pro
Transitioning from IT Security to Full-Time Bug Bounty: A Realistic Roadmap
How to Automate Your Recon with Subfinder, httpx, and Nuclei
From Zero to P1: A Writeup of a Critical Authentication Bypass

Notes from the front lines of bug hunting.