Site icon Filestack Blog

How to Implement Bulk File Deletion with Filestack in Node JS

bulk_delete_thumbnail

A common situation in many applications is that a user begins uploading a file but never completes the process. They may close the tab, abandon the form, or simply not submit. The file is uploaded to Filestack but never gets attached to any content or record in your database.

These abandoned uploads, along with files from deleted posts, replaced images, or removed user accounts, continue to occupy storage and generate costs month after month.

This guide shows you how to identify these unused files, build a reliable bulk deletion tool, and automate the cleanup process so you can reduce your Filestack storage expenses effectively.

Why Storage Cleanup Is Important

Filestack does not automatically remove files. Every handle you create remains in storage until you delete it. Over time, abandoned and orphaned files can represent a large portion of your total storage usage. Regular cleanup often reduces storage costs by 30 to 60 percent, depending on your application’s activity level.

Understanding the Filestack File API for Deletion

The File API delete endpoint handles one file at a time:

DELETE https://www.filestackapi.com/api/file/{HANDLE}

This operation requires security credentials in the form of a policy and signature. You can also use the parameter skip_storage=true if you want to remove the handle from Filestack while keeping the file in your underlying storage provider (S3, GCS, etc.).

Building the Cleanup Solution

1. Track Files in Your Database

The first step to take is to maintain a dedicated table to record every upload. This becomes your source of truth for identifying files that can be safely deleted.

Example table structure:

SQL:

CREATE TABLE files (

  id SERIAL PRIMARY KEY,

  filestack_handle TEXT UNIQUE NOT NULL,

  user_id INTEGER,

  related_entity_id INTEGER,

  uploaded_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,

  last_used_at TIMESTAMP,

  deleted_at TIMESTAMP

);

Save the handle and relevant metadata immediately after every successful upload.

2. Policy and Signature Generation

Create a helper function to generate the required security policy for deletion.

JavaScript:

// utils/filestack.js

const crypto = require('crypto');




function createDeletePolicy(secretKey, minutesValid = 20) {

  const policy = {

    expiry: Math.floor(Date.now() / 1000) + (minutesValid * 60),

    call: ['remove']

  };


  const policyB64 = Buffer.from(JSON.stringify(policy)).toString('base64url');

  const signature = crypto

    .createHmac('sha256', secretKey)

    .update(policyB64)

    .digest('base64url');

  return { policy: policyB64, signature };

}

module.exports = { createDeletePolicy };

3. Single File Deletion Function

JavaScript:

// services/filestackService.js

async function deleteFilestackFile(handle, apiKey, secretKey) {

  const { policy, signature } = createDeletePolicy(secretKey);

  const url = `https://www.filestackapi.com/api/file/${handle}?key=${apiKey}&policy=${policy}&signature=${signature}`;

  const res = await fetch(url, { method: 'DELETE' });

  if (!res.ok) {

    const errorText = await res.text();

    throw new Error(`Delete failed for ${handle}: ${res.status} ${errorText}`);

  }

  return true;

}

4. Bulk Deletion Function

JavaScript:

// services/cleanupService.js

const pLimit = require('p-limit');

async function bulkDelete(handles, apiKey, secretKey, maxConcurrent = 6) {

  const limit = pLimit(maxConcurrent);

  let success = 0;

  let failed = 0;

  const errors = [];


  const tasks = handles.map(handle => 

    limit(async () => {

      try {

        await deleteFilestackFile(handle, apiKey, secretKey);

        success++;

        // Update your database here: set deleted_at

      } catch (e) {

        failed++;

        errors.push({ handle, error: e.message });

      }

    })

  );


  await Promise.all(tasks);

  return { success, failed, errors };

}

5. Identifying Abandoned Uploads

Use a query like this to find files that were never attached to any content:

SQL:

SELECT filestack_handle 

FROM files 

WHERE deleted_at IS NULL 

  AND (last_used_at IS NULL OR last_used_at < NOW() - INTERVAL '45 days')

  AND related_entity_id IS NULL

LIMIT 500;

Automating Cleanup with a Cron Job

The most effective approach is to run the cleanup automatically on a schedule. Here is a complete example using node-cron.

First, install the package:

Bash:

npm install node-cron

Then create the scheduled job:

JavaScript:

// jobs/filestackCleanup.js

const cron = require('node-cron');

const { bulkDelete } = require('../services/cleanupService');

const db = require('../config/db'); // your database connection

const { apiKey, secretKey } = require('../config/filestack');


cron.schedule('0 3 * * 0', async () => {  // Every Sunday at 3:00 AM

  console.log('Starting scheduled Filestack cleanup for abandoned uploads...');

  const dryRun = process.env.CLEANUP_DRY_RUN === 'true';

  try {

    const queryResult = await db.query(`

      SELECT filestack_handle 

      FROM files 

      WHERE deleted_at IS NULL 

        AND (last_used_at IS NULL OR last_used_at < NOW() - INTERVAL '45 days')

        AND related_entity_id IS NULL

      LIMIT 800;

    `);

    const handles = queryResult.rows.map(row => row.filestack_handle);

    if (handles.length === 0) {

      console.log('No abandoned files found.');

      return;

    }

    console.log(`Found ${handles.length} abandoned files.`);

    if (dryRun) {

      console.log('Dry run mode: No files will be deleted.');

      return;

    }

    const result = await bulkDelete(handles, apiKey, secretKey, 6);

    console.log(`Cleanup completed. Success: ${result.success}, Failed: ${result.failed}`);

    // Optional: update database records for successfully deleted files

  } catch (error) {

    console.error('Cleanup job failed:', error);

  }

});

console.log('Filestack cleanup cron job scheduled.');

Include this file in your main application startup so the cron job registers automatically.

Additional Recommendations

By implementing this process, you can systematically remove abandoned files and keep your Filestack storage costs under control without manual effort each month.

Exit mobile version