PhishingNet: privacy policy and project documentation
How to get real inbox data out of a small team, safely, and turn it into something you can plot — and, more importantly, into labelled examples that make the detector better. Everything described here already exists in this repo; nothing is hand-waved.
A five-person pilot produces a few thousand rows. That is plenty for some questions and hopeless for others. Pick the ones it can actually answer before you start:
It can answer these
It cannot answer these (see the caveats): detection rate against real attacks, whether the tool prevented anything, or anything with a denominator of “all email received”.
Write your questions down on day one. It stops the pilot drifting into collecting numbers nobody looks at.
One row per email opened while the extension was on, plus one row per correction.
| Stored | Not stored |
|---|---|
| timestamp, team code, pseudonymous participant id | any name, email address, or local-part |
| webmail client (gmail / outlook) | the subject |
| verdict, risk score | the message body |
| classifier probability (raw + gated) | any link or URL |
| which attributes fired, and their scores | any attachment or filename |
| corroboration outcome, sender-trust tier | anything a person wrote |
the sender’s registrable domain (atlassian.net) |
the sender’s full address |
| a salted one-way hash of the normalised body | the body the hash was made from |
The body hash exists for exactly one purpose: answering “did this same message also reach someone else on the team?”. Digits are normalised out first, so per-recipient tracking ids do not break the match. It is salted per deployment and truncated, so it is useless anywhere but inside your own database.
The boundary is enforced, not just documented:
scripts/seed_demo_telemetry.py runs a privacy audit that fails loudly if any
message content or unexpected column reaches the database. Run it after any change
to server/telemetry.py.
Collection is off unless both are true: the service was started with
TELEMETRY_ENABLED=1, and the participant typed a team code into the popup.
Chrome (x5) your analyzer service you
┌────────────────┐ ┌──────────────────────┐ ┌────────────┐
│ PhishingNet │ POST │ /analyze │ │ /dashboard │
│ content.js │──/analyze──▶│ score the email │ │ charts │
│ + team code │ (+team, │ ├─ return verdict │───────▶│ tables │
│ │◀─participant)│ └─ telemetry.record│ └────────────┘
│ banner shows │ verdict + │ /feedback │ ┌────────────┐
│ ✓ Correct │──event_id──▶│ store correction │ │ /export │
│ ✕ Wrong │ POST │ SQLite (one file) │───────▶│ JSON rows │
└────────────────┘ /feedback └──────────────────────┘ └────────────┘
The important property: telemetry rides along on the request that was already happening. The analyzer needs the email text in order to score it; enabling the pilot only tells it to keep the verdict it just produced. No second copy of anything is transmitted, and there is no separate tracking call.
SQLite needs a file that survives restarts. Render’s filesystem is ephemeral — without a persistent disk your data vanishes on the next deploy, which for a four-week pilot is the difference between a result and nothing. Either:
Option A — attach a Render disk (recommended). In the service’s Disks tab add
a 1 GB disk mounted at /var/data, then set the env vars below. render.yaml
carries the disk declaration already.
Option B — run the collector yourself. Keep the hosted analyzer as it is, and run a second instance on a machine you control (a spare laptop, a small VPS) that the five browsers point at via the popup’s Analyzer URL field. Simplest if you would rather the data never sat on a third-party host at all.
Option C — accept the loss and export often. Run GET /export on a schedule and
keep the JSON. Fine for a two-week pilot, risky for longer.
TELEMETRY_ENABLED=1 # nothing is recorded without this
TELEMETRY_DIR=/var/data # the mounted disk (Option A)
TELEMETRY_SALT=<32 random hex chars> # set it, back it up, never rotate mid-pilot
Generate the salt once: python -c "import secrets; print(secrets.token_hex(16))".
If you leave it unset the service generates one and stores it, which is fine — but
losing it means campaign grouping resets.
TELEMETRY_ENABLED=1 TELEMETRY_DIR=./pilot-data \
.venv/Scripts/python.exe -m scripts.seed_demo_telemetry --reset --days 21
TELEMETRY_ENABLED=1 TELEMETRY_DIR=./pilot-data \
.venv/Scripts/python.exe -m uvicorn server.app:app --port 8008
# open http://127.0.0.1:8008/dashboard
That seeds a simulated five-person fortnight from the real-world fixtures, runs the
privacy audit, and gives you a populated dashboard. Look at it now, decide
whether it answers your questions, and change it while it costs nothing. Then
rm -rf ./pilot-data before going live so demo rows never mix with real ones.
/stats, /export and /dashboard are unauthenticated in this repo. For five
people that may be acceptable — there is no message content behind them — but the
sending domains and campaign timing are still your team’s business. Put them behind
Render’s IP allow-list, a reverse proxy with basic auth, or run Option B. Do not
skip this step and then forget about it.
The published Chrome Web Store version does not contain the pilot code, and you should keep it that way: adding telemetry to the public listing means a privacy-policy change and a fresh review cycle, and this repo already has v1.2.0 (Outlook support) waiting to be submitted. Do not entangle the two.
Run the pilot on an unpacked build:
.venv/Scripts/python.exe -m scripts.build_extension_zip # or just use extension/ directly
Each participant: chrome://extensions → enable Developer mode → Load
unpacked → select the extension/ folder. Five installs, five minutes, no review
cycle, and you can ship a fix the same afternoon rather than waiting a week.
If someone already has the store version, have them disable it for the duration — two copies will stack two banners on every email.
In the extension popup:
northgate. Typing it
is what turns collection on; clearing it turns it off immediately.The popup then shows the participant id the browser generated. That id is the only thing tying rows together; nothing derives it from the person.
Gmail and Outlook are not equivalent. Gmail exposes its own SPF/DKIM verdict, which the extension forwards, so the analyzer can clear genuine transactional mail. Outlook exposes nothing of the kind, so the same email is judged on wording alone and scores more harshly. Record which client each person uses — the dashboard breaks this out — and do not compare their false-positive rates directly.
Have it properly, in person, before anyone installs. Five people who feel surveilled will stop opening mail in the browser and your data will be worthless.
Cover, honestly:
participants block from telemetry.stats().Write the answers down in one short page and give everyone a copy. If your organisation has anyone responsible for data protection, this is a five-minute conversation with them now rather than an incident later.
Install, configure, and change nothing for one week. You need a picture of normal traffic before you start tuning, or you will have no idea whether a later change helped. Tell people to use the ✓/✕ buttons freely, especially the ✕.
At the end of week 0, record the numbers that matter:
curl "https://<your-service>/stats?days=7&team=northgate" > week0.json
Keep the JSON. The flagged-rate from week 0 is what every later week is compared to.
Once a week, ten minutes:
curl "https://<your-service>/export?team=northgate" > pilot-final.json
Then turn off TELEMETRY_ENABLED, and delete the data on the date you promised.
GET /dashboard — four tiles, two charts, two tables.
Every chart has a Table view toggle, and /stats returns the same numbers as
JSON if you would rather plot them in a notebook.
This is the part that makes the pilot worth running. A correction in the queue becomes a permanent regression test:
tests/fixtures/realworld_cases.py: strip anything
personal, keep the structure and wording that caused the verdict, and write the
why field explaining what it is guarding against..venv/Scripts/python.exe -m evaluate_realworld
It exits non-zero on any regression, so it can gate a deploy.
AMBIENT in lexicons.py. If the classifier is drifting mid-range, that is
classifier_gate. If a real scam scored low, look for a mechanism that
distinguishes it — the sender’s domain, where its links go, what it asks for —
rather than adding another keyword.pytest -q) and evaluate_realworld before deploying.
Both are cheap; a regression that reaches five colleagues is not.Twenty real corrections are worth more than another 50,000 rows of 2000s corpus. That is the whole reason to run a pilot rather than tune against the test set.
Be straight about this in any write-up.
None of this makes the pilot pointless — it makes it a product evaluation (does this tool help these people without annoying them?) rather than an epidemiological one. That is the question worth answering anyway.