Digital Mailroom Automation With Envelope OCR

A virtual mailbox company gets a sack of mail every morning for hundreds of customers who all share one street address. Someone scans each envelope, reads the name and the box number off the screen, and types them into the system that tells the customer they have mail. That typing is the step digital mailroom automation removes. The scan already holds the answer.

 

Put envelope_ocr in front of that scan’s handle, and the response names the parts:

{
  "recipient_address": {
    "text": "Tomasz Wisniewski\n1200 Commerce St #2045\nTacoma, WA 98402"
  },
  "recipient_name": "Tomasz Wisniewski",
  "sender": "A. Wisniewska\n19 Birch Hollow Rd\nBend, OR 97701"
}

Mailbox 2045 belongs to Tomasz Wisniewski, so this envelope can go to his mailbox without anyone typing. What turns that response into a delivery is a matching step: compare the name and number with your mailbox holders, and queue what doesn’t fit. It’s document process automation with one document type and one decision.

What is digital mailroom automation?

Digital mailroom automation is scanning incoming paper mail and using OCR to read who each item is for, so software routes it to the right person, mailbox or workflow without anyone typing the details. Mail scanning creates the image, OCR reads the sender and the recipient, and anything the software can’t match goes to a review queue for a person.

If you build virtual mailbox software, or run a coworking space or managed office where many customers share one street address, this tutorial builds that flow with Filestack Envelope OCR and a small Node.js service.

Key takeaways

• envelope_ocr returns sender, recipient_name and recipient_address.text as named fields, so the code works with the recipient block directly instead of with positions on the page.
• Match on the mailbox number in the address first, and check the name against that box’s holders. A number and a name that point at different customers go to review.
• When the name sits apart from the address lines, the recipient block starts at the street line. Plain ocr on the same scan still reads the name.
• A scan that isn’t an envelope returns HTTP 200 with every field empty, which the router treats as a review item.
• The upload webhook can arrive more than once, so key each record by handle and read each envelope once.

What the mailroom has to decide for each envelope

Each scan needs one answer: which mailbox, or a person looks at it. At a shared address that answer comes from two things on the envelope, the mailbox number and the name.

The number is the strongest signal. It’s how the customer was told to write their address, as #1187, PMB 742 or Box 3310 after the street. The name confirms it. A number with the wrong name can be a typo or mail for someone who moved out, and both need a person.

Envelopes without a number are common too. Businesses write their own name and the suite, and a law firm writes “Attn:” and a partner’s name under the company. Those match on the name alone, and only when exactly one holder fits.

Ways to read the envelope

The Envelope OCR fields cover the full response and how it compares with plain OCR.

Get each scan into storage

The scanning station uploads each envelope and keeps going. These examples use Node.js 20.6 or later, with no packages to install. upload-scan.mjs runs on the station, signs a short policy with the app secret, and sends each file to the store endpoint of the file uploader:

import { createHmac } from 'node:crypto';
import { readFile } from 'node:fs/promises';
import { basename } from 'node:path';

// Runs on the scanning station, which holds the app secret
const expiry = Math.floor(Date.now() / 1000) + 300;
const policy = Buffer.from(JSON.stringify({ call: ['pick', 'store'], expiry }))
  .toString('base64')
  .replace(/\+/g, '-')
  .replace(/\//g, '_');
const signature = createHmac('sha256', process.env.FILESTACK_APP_SECRET).update(policy).digest('hex');

for (const path of process.argv.slice(2)) {
  const params = new URLSearchParams({
    key: process.env.FILESTACK_API_KEY,
    policy,
    signature,
    filename: basename(path),
  });
  const res = await fetch(`https://www.filestackapi.com/api/store/S3?${params}`, {
    method: 'POST',
    headers: { 'Content-Type': 'image/png' },
    body: await readFile(path),
  });
  if (!res.ok) throw new Error(`${basename(path)}: ${res.status} ${await res.text()}`);
  const { url } = await res.json();
  console.log(basename(path), url.split('/').pop());
}

The three values go in a .env file next to the scripts. The API key and the app secret come from the Developer Portal, and the webhook secret appears in the next step:

FILESTACK_API_KEY=YOUR_API_KEY
FILESTACK_APP_SECRET=YOUR_APP_SECRET
FILESTACK_WEBHOOK_SECRET=YOUR_WEBHOOK_SECRET

node --env-file=.env upload-scan.mjs scans/*.png

Change the Content-Type header if your scanner saves JPEG. The station doesn’t call OCR. It stores the file, and the upload event starts everything else.

Read each envelope when the upload webhook arrives

Every file uploaded to the application triggers an fp.upload event. In the Developer Portal, open Configuration, then Webhooks, enter your server’s URL, choose File upload as the type and add it. Then click Create in that row’s Secret column, and use the value it shows as FILESTACK_WEBHOOK_SECRET. A webhook with no secret delivers without the FS-Signature header, and the server rejects those with a 401.

 

The event says which file arrived. The handle is the last segment of text.url:

{
  "id": "aac34f3e-2b38-40c0-9397-9ca9091cbd42",
  "action": "fp.upload",
  "timestamp": 1791202793,
  "text": {
    "url": "https://cdn.filestackcontent.com/HANDLE",
    "client": "Computer",
    "type": "image/png",
    "filename": "scan.png",
    "size": 46045
  }
}

filestack.mjs holds everything that talks to Filestack: the signed reads and the webhook check. Envelope OCR runs on Start, Grow and Scale, and every call needs a signed policy with read and convert. Without a policy the response is 403 security required for tasks: envelope_ocr, and a policy missing convert returns 403 ... call is not allowed: (call: convert).

import { createHmac, timingSafeEqual } from 'node:crypto';

const CDN = 'https://cdn.filestackcontent.com';

function signedUrl(task, handle) {
  const expiry = Math.floor(Date.now() / 1000) + 300;
  const policy = Buffer.from(JSON.stringify({ call: ['read', 'convert'], handle, expiry }))
    .toString('base64')
    .replace(/\+/g, '-')
    .replace(/\//g, '_');
  const signature = createHmac('sha256', process.env.FILESTACK_APP_SECRET)
    .update(policy)
    .digest('hex');
  return `${CDN}/security=p:${policy},s:${signature}/${task}/${handle}`;
}

async function getJson(task, handle) {
  const res = await fetch(signedUrl(task, handle));
  if (!res.ok) throw new Error(`${task} ${handle}: ${res.status} ${await res.text()}`);
  return res.json();
}

export const readEnvelope = (handle) => getJson('envelope_ocr', handle);
export const readText = async (handle) => (await getJson('ocr', handle)).text;

export function verifyWebhook(raw, timestamp, signature) {
  if (!timestamp || !signature) return false;
  const expected = createHmac('sha256', process.env.FILESTACK_WEBHOOK_SECRET)
    .update(`${timestamp}.${raw}`)
    .digest('hex');
  const a = Buffer.from(expected);
  const b = Buffer.from(signature);
  return a.length === b.length && timingSafeEqual(a, b);
}

The signature covers FS-Timestamp, a full stop and the raw body, exactly as received. Parse the JSON before checking, and the bytes you hash are no longer the bytes that were signed.

server.mjs receives the event, answers straight away, and sorts the envelope after the response has gone:

import { createServer } from 'node:http';
import { readEnvelope, readText, verifyWebhook } from './filestack.mjs';
import { decide } from './route.mjs';

// Stand-in store. Replace with your database.
const mail = new Map();

async function sort(handle) {
  const fields = await readEnvelope(handle);
  let decision = decide(fields);
  if (decision.action === 'need_name') {
    decision = decide(fields, await readText(handle));
  }
  mail.set(handle, { handle, fields, decision, at: new Date().toISOString() });
  console.log(handle, decision.action, decision.box ?? decision.reason);
}

createServer(async (req, res) => {
  if (req.method !== 'POST' || req.url !== '/webhooks/filestack') {
    res.writeHead(404).end();
    return;
  }

  let raw = '';
  for await (const chunk of req) raw += chunk;
  if (!verifyWebhook(raw, req.headers['fs-timestamp'], req.headers['fs-signature'])) {
    res.writeHead(401).end('invalid signature');
    return;
  }

  const event = JSON.parse(raw);
  const handle = event.text?.url?.split('/').pop();
  if (event.action !== 'fp.upload' || !event.text?.type?.startsWith('image/') || mail.has(handle)) {
    res.writeHead(204).end();
    return;
  }

  mail.set(handle, { handle, decision: { action: 'reading' } });
  res.writeHead(200).end();

  sort(handle).catch((err) => {
    mail.set(handle, { handle, decision: { action: 'review', reason: 'read failed' } });
    console.error(handle, err.message);
  });
}).listen(8787, () => console.log('mailroom listening on :8787'));

Three things in that handler keep the bill and the queue clean:

• One read per envelope. The envelope_ocr response is Cache-Control: private, and every request runs the task again. 1,000 envelopes a month are included, then $0.03 an envelope. So the fields are stored against the handle the first time, and nothing reads the scan again.
• A delivery can arrive twice. When your server answers with anything other than 200, 201 or 204, we retry the delivery 5 minutes, 30 minutes and 12 hours later. The handle is recorded before the response goes out, so a second delivery of the same upload gets a 204 and costs nothing.
• Only image uploads are read. Other uploads on the same application, such as PDFs from another part of the product, get a 204 and are left alone, and so do other event types. Each OCR read sends an fp.converse event to the File conversion webhook if one is set, so point that webhook at a different URL.

If the read fails, the envelope lands in review with the reason, rather than sitting in a reading state nobody looks at.

Match the recipient to a mailbox holder

route.mjs turns the three fields into a decision. It looks for a mailbox number in the recipient block, takes the name lines above the street line, and checks them against your holders. The holder list here is a constant. In production it’s a query against your customer table.

// Your mailbox holders. In production this is a query against your customer table.
const MAILBOXES = [
  { box: '1187', names: ['Priya Raman', 'Raman Design Studio'] },
  { box: '2045', names: ['Tomasz Wisniewski'] },
  { box: '3310', names: ['Okafor & Lindqvist Architects', 'Chidi Okafor', 'Elin Lindqvist'] },
  { box: '742', names: ['Mei-Lin Zhou'] },
];

const normalize = (s) =>
  s.toLowerCase()
    .replace(/\b(attn|c\/o|dr|mr|mrs|ms)\b\.?:?/g, ' ')
    .replace(/[^a-z0-9]+/g, ' ')
    .trim();

// "#1187", "PMB 742" or "Box 3310" in the address block
function mailboxNumber(lines) {
  for (const line of lines) {
    const m = line.match(/(?:#|\bPMB\s*|\bBox\s+)0*(\d{1,5})\b/i);
    if (m) return m[1];
  }
  return null;
}

const looksLikeStreet = (line) => /^\d/.test(line.trim());

export function decide(fields, pageText) {
  const address = fields.recipient_address?.text ?? '';
  if (!address && !fields.recipient_name && !fields.sender) {
    return { action: 'review', reason: 'not an envelope' };
  }

  const lines = address.split('\n').map((l) => l.replace(/\|+/g, '').trim()).filter(Boolean);
  const box = mailboxNumber(lines);

  // Name lines are the ones above the street line
  const street = lines.findIndex(looksLikeStreet);
  let names = (street === -1 ? lines.slice(0, -1) : lines.slice(0, street)).map(normalize);
  if (names.length === 0) {
    if (!pageText) return { action: 'need_name' };
    const senderLines = new Set((fields.sender ?? '').split('\n').map(normalize));
    names = pageText.split('\n').map(normalize).filter((n) => n && !senderLines.has(n));
  }

  const named = MAILBOXES.filter((m) => m.names.some((n) => names.includes(normalize(n))));

  if (box) {
    const holder = MAILBOXES.find((m) => m.box === box);
    if (!holder) return { action: 'review', reason: `no mailbox ${box}` };
    if (named.includes(holder)) return { action: 'deliver', box };
    if (named.length) return { action: 'review', reason: `mailbox ${box} but addressed to the holder of ${named[0].box}` };
    return { action: 'review', reason: `name is not on mailbox ${box}` };
  }

  if (named.length === 1) return { action: 'deliver', box: named[0].box };
  if (named.length > 1) return { action: 'review', reason: 'more than one holder matches' };
  return { action: 'review', reason: 'no holder matches' };
}

The router reads the name from every line above the street, not only from recipient_name. On a business envelope with “Okafor & Lindqvist Architects” over “Attn: Elin Lindqvist”, recipient_name is the Attn line and the company is the line above it. Both are holders of box 3310, so either one matches. normalize drops the “Attn:”, the titles and the punctuation, so “Mei-Lin Zhou” on an envelope matches “Mei Lin Zhou” in your table. Pipes are stripped too, because a routing barcode printed across the last address line can come back as |||| on the end of it.

Names are matched exactly after that cleanup. A misspelt name goes to review rather than to the closest holder, which is the right default for mail. Add fuzzy matching only for the number-plus-name case, where the box has already narrowed the choice to one customer.

When the recipient block starts at the street line

When the recipient’s name is printed apart from the address lines, recipient_address starts at the street line and recipient_name holds that street line. This envelope is one of them:

 

{
  "recipient_address": {
    "text": "1200 Commerce St Ste 400 #1187\nTacoma, WA 98402"
  },
  "recipient_name": "1200 Commerce St Ste 400 #1187",
  "sender": "PO Box 3900\nSpokane, WA 99220"
}

The box number is there, so the envelope could be routed on the number alone. That is the risky case. An envelope addressed to Priya Raman with Tomasz Wisniewski’s box number would be delivered to him without anyone noticing. So when no name line sits above the street line, decide returns need_name, and sort reads the page with plain OCR API text and decides again. On this envelope the text includes Priya Raman on its own line, which is a holder of box 1187, and the envelope is delivered. Lines that belong to the sender are removed first, so a customer posting a letter to another customer can’t match on the return address.

That second read uses one OCR unit from the 5,000 a month on the same plans. It only runs for envelopes whose recipient block starts at the street.

What lands in the review queue

For the eight scans in this example, the router gives these answers:

A file with no envelope in it, like the receipt, returns HTTP 200 with {"recipient_address":{"text":""},"recipient_name":"","sender":""}. Nothing fails, so the router checks the fields, not the status code. Photos, letters scanned without their envelope and envelopes scanned face down all arrive the same way.

recipient_address.text is what was written on the envelope, not a validated postal address. For a mailroom that only needs to find the customer at its own address, that is enough. Forwarding the mail onwards is a separate step, and the forwarding address belongs to the customer record, not to the scan.

Run it locally

Start the server:

node --env-file=.env server.mjs

The portal needs a public URL to deliver to, so for local work, sign a test event yourself. test-event.mjs sends the server an fp.upload event for a handle, signed with your webhook secret the same way we sign real deliveries:

import { createHmac, randomUUID } from 'node:crypto';

// Sends the server an fp.upload event for a handle, signed like a real delivery
const handle = process.argv[2];
const timestamp = Math.floor(Date.now() / 1000);
const body = JSON.stringify({
  id: randomUUID(),
  action: 'fp.upload',
  timestamp,
  text: { url: `https://cdn.filestackcontent.com/${handle}`, type: 'image/png', filename: 'scan.png' },
});
const signature = createHmac('sha256', process.env.FILESTACK_WEBHOOK_SECRET)
  .update(`${timestamp}.${body}`)
  .digest('hex');

const res = await fetch('http://127.0.0.1:8787/webhooks/filestack', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json', 'FS-Timestamp': String(timestamp), 'FS-Signature': signature },
  body,
});
console.log(res.status);

Upload a scan with upload-scan.mjs, then pass the handle it prints:

node --env-file=.env test-event.mjs HANDLE

The test script prints 200, and the server prints the decision a moment later, such as deliver 1187 or review no holder matches. Send the same handle again and the script prints 204, because that envelope has already been read. A request with a wrong signature gets a 401. To check the signing code against a known answer, the upload webhooks reference publishes an fp.upload body, timestamp and signature for the secret secret, and verifyWebhook returns true for it.

Once the server is on a public URL, point the File upload webhook at https://YOUR_DOMAIN/webhooks/filestack and scan a real envelope. The Earth Class Mail case study describes a virtual mailbox business that runs its physical mail through this kind of pipeline. Next, replace MAILBOXES with a lookup against your customer table and put the review queue in front of the person who used to type every envelope, so they only see the ones the router couldn’t place. That way, digital mailroom automation handles the routine envelopes, and people handle only the exceptions.

Digital mailroom automation FAQ

What is digital mailroom automation?

It’s scanning incoming paper mail and using OCR to read the recipient from each envelope, so software sends the item to the right mailbox and passes anything it can’t match to a person.

How does Envelope OCR work?

Envelope OCR reads a scanned envelope and returns the sender, the recipient name and the recipient address as separate JSON fields, so your code matches the recipient directly instead of working out where text sits on the page.

Can Envelope OCR read handwritten envelopes?

Yes. The handwritten envelope in this tutorial, addressed to Tomasz Wisniewski at mailbox 2045, returned the correct recipient name, mailbox number and address.

What happens when the mailbox number and the name don’t match?

The envelope goes to the review queue. A number and a name that point at different customers can be a typo or mail for someone who has moved out, so a person decides.

How much does Envelope OCR cost?

Envelope OCR is included on the Filestack Start, Grow and Scale plans with 1,000 envelopes a month, and each envelope above that costs $0.03. The plain OCR fallback uses a separate allowance of 5,000 OCR units a month on the same plans.

Read More →