Image Captioning | Create an Image Accessibility Viewer with Filestack

Does your app truly cater to all? More than 2.2 billion people worldwide have some visual impairment or the other. Ensuring your app is designed to conform to accessibility standards is necessary. Image captioning is one of those ways to do it best. It’s a technology that automatically creates image caption, thus making your app more inclusive.

This blog will explain why image captioning is vital, how it improves accessibility, and why you should consider this factor. We will also review real websites that use image caption features and indicate their benefits for user engagement.

This blog post provides a step-by-step guide for adding image accessibility to your apps, focusing on using Filestack in a React JS environment. Let’s start with the basics.

What is image captioning?

Image captioning is an example of automatic picture description creation technology. It uses machine learning capability to study an image and create a caption that explains what the picture represents. This makes it easier for everyone, especially those with visual impairments, because they can connect what they see with what they read.

For example, Instagram and Facebook employ this adaptive assistive imagery service. When someone uploads a photograph onto these platforms, they can generate descriptions automatically describing them. Visually impaired people can understand what a picture portrays by reading the description aloud through screen readers.

Another example of AI-based captioning comes from YouTube. The system does not just provide captions for spoken words; it also describes actions in the video clip. Thus, anyone can easily watch such content regardless of sight problems.

How does image captioning play an essential role in achieving accessibility?

Captioning images aids accessibility in various ways, including providing descriptive texts for images required by persons with vision disabilities.

Image captioning is also beneficial for users with cognitive impairments. Some may find it difficult to comprehend complex images, but simple captioning makes the image’s meaning understandable.

For instance, a news website can use image captioning to describe photos in its articles. This way, even people with disabilities can follow the story without problems.

Moreover, e-commerce sites can describe their product images using captions. Such descriptions help people with disabilities make better choices when shopping.

Captioning provides a more inclusive experience for businesses. It not only ensures compliance with the law but also shows an intention to serve all customers. In this way, the message in your content is accessible to every person.

Also read: Usability & Web Accessibility

What are the benefits of adding accessibility to your applications?

Making your apps accessible also has many advantages. By enabling it as a touch point for persons with different kinds of disabilities, you will be able to increase its market share among such users;

Microsoft’s Office suite added features like screen reader support and keyboard shortcuts. This made it possible for visually impaired people or people with motor difficulties to use the software, resulting in increased adoption rates as well.

Accessibility benefits everyone. For example, voice commands and text-to-speech are helpful not just for those with a disability but for everyone at large. Apple’s VoiceOver was initially designed for blind people but was used by many iPhone owners since this technology allows for the hands-free operation of these gadgets.

Third, the law requires that you be able to access something. This means that accessibility is a requirement in many countries, such as the US and EU. Failure to follow these regulations can lead to fines or even being sued. For example, Target was taken to court over its website, which lacked access features for PWDs but was later fixed.

Lastly, accessibility enhances your brand image. It indicates that your business has concerns about inclusivity and corporate citizenship. Google Maps also unveiled wheelchair-accessible routes for users with disabilities. This not only meets user needs but also strengthens Google’s reputation.

Making your app accessible is more than a legal requirement. It’s a wise decision that improves user experience, increases your audience, and creates a positive brand.

What are real-life examples of websites that are using accessibility features?

Several websites now have accessibility options in place for everyone to use them, including disabled people. Here are some examples:

1. BBC

The BBC website is superbly accessible. Users can change text sizes, switch to high-contrast colors, or navigate using a keyboard interface. These characteristics help individuals who suffer from visual challenges and those who may not be capable of regular navigation.

2. Apple

The accessibility features are an integral part of the Apple website. VoiceOver reads text aloud for users with visual impairments on the site, supports keyboard shortcuts, and offers comprehensive accessibility guidelines for its products.

3. Amazon

Amazon has introduced a range of features that enhance accessibility on its platform, including alternative text, keyboard navigation, and screen reader compatibility, making it easy for people with visual and motor disabilities to shop online.

4. Microsoft

There are quite several accessibilities supported by Microsoft’s web page content. Text can be resized; videos come with closed captions enabled, whereas navigating links within the page works well with the keyboard-only method, too, thus accommodating various assistive technologies in use today.

How do you integrate image accessibility within your apps?

First, you should decide which technology to integrate image accessibility with your apps. This is the most crucial step that most people ignore.

The most reliable way to add image accessibility within your apps is to use an API. Many APIs are available on the market. However, Filestack is the simplest API for such features. The best part? Filestack also comes with comprehensive documentation and secure integration, so you can rely on it.

Let’s explore Filestack API. Next, we will show you the most straightforward integration steps to create an image accessibility viewer.

Filestack

Filestack’s Image Captioning is an intelligent tool that automatically creates text descriptions for your images. It uses advanced attention networks to analyze the image and generate a caption. This feature is valuable for improving accessibility and user experience.

The service is available through Filestack’s Processing API. You can use it as a synchronous operation with the “caption” task. To use this task, you need a security policy and signature. Once set up, you can quickly get captions for your images.

For example, after uploading an image, you can retrieve its caption using the handle:

https://cdn.filestackcontent.com/security=p:<POLICY>,s:<SIGNATURE>/caption/<HANDLE>

You can also combine the caption task with other tasks, like resizing the image:

https://cdn.filestackcontent.com/security=p:<POLICY>,s:<SIGNATURE>/resize=h:1000/caption/<HANDLE>

You can even use the caption task with an external URL:

https://cdn.filestackcontent.com/<FILESTACK_API_KEY>/security=p:<POLICY>,s:<SIGNATURE>/caption/<EXTERNAL_URL>

The response from Filestack’s Image Captioning is in JSON format. It includes a “caption” parameter with the generated text. For example:

{

    "caption": "A close up of a bird"

}

This feature can also be integrated into Filestack Workflows for automated processing.

Let’s show you the integration of this feature.

Integrating image accessibility into a React JS app

First, create an account at Filestack. Click on the Sign Up Free button at the top right corner of the Filestack website.

It will show up as a form. You should fill out that form and create an account. Every time you log in to Filestack, you must enter a verification code. You will receive this code at your registered email.

After logging in to your account, you should see a dashboard. At the top right corner, you will see an API key. Get this API key and keep it somewhere safe.

Next, you should get the policy and signature. For this purpose, click on the security tab in the left column of the dashboard. Then, click on the Policy & Signature button.

Set an expiry date, check all boxes, and copy the signature and policy by scrolling down. The next step is to create a React app.

Building the App

Open the command prompt on your PC. Make sure that you have installed React JS on your system. Enter the commands below to create a React app.

npx create-react-app image-caption-generator

cd image-caption-generator

Next, you should install the Filestack library inside the app you just created.

npm install filestack-js

Next, open the React JS app inside the visual studio code.

code .

The coding part starts now. You should open the App.js component inside the src directory and enter the code given below:

1. Import Statements

   import React, { useState } from 'react';

   import { Client } from 'filestack-js';

   import './App.css';

The `React` module is imported to enable the creation of React components.
useState is used to create state variables in functional components.
The `Client` class from the Filestack SDK is imported to interact with Filestack’s services.
‘./App.css’ is the CSS file `App.css` is imported to style the component.

2. Component Declaration

   const App = () => {

React component named `App` is declared using an arrow function.

3. State Variables

   const [imageUrl, setImageUrl] = useState('');

   const [caption, setCaption] = useState('');

   const [showCaptionContainer, setShowCaptionContainer] = useState(false);

   const [copyMessageVisible, setCopyMessageVisible] = useState(false);

imageUrl stores the URL of the uploaded image.
Caption holds the generated caption for the uploaded image.
showCaptionContainer is a boolean flag to control the visibility of the caption display container.
copyMessageVisible is a boolean flag to control the visibility of the “caption copied” message.

4. Filestack Client Initialization

   const apiKey = 'ADD-YOUR-API-KEY-HERE'; // Replace with your Filestack API key

   const client = new Client(apiKey);

apiKey is the API key for authenticating requests to Filestack.
client is an instance of the Filestack `Client`, initialized with the API key, to interact with Filestack services.

5. openPicker Function

   const openPicker = () => {

     client.picker({

       onUploadDone: (result) => {

         const file = result.filesUploaded[0];

         setImageUrl(file.url);

         generateCaption(file.handle);

       },

     }).open();

   };

openPicker is a function that opens the Filestack file picker.
client.picker() initiates the Filestack picker. The `onUploadDone` callback is triggered when a file is successfully uploaded.
setImageUrl updates the `imageUrl` state with the uploaded file’s URL.
generateCaption(file.handle) calls the `generateCaption` function with the file handle to generate a caption for the uploaded image.

6. generateCaption Function

const generateCaption = (handle) => {

     const policy = 'eyJleHBpcnkiOjE3MjQyNzQwMDAsImNhbGwiOlsicGljayIsInJlYWQiLCJzdGF0Iiwid3JpdGUiLCJ3cml0ZVVybCIsInN0b3JlIj3Q=='; // Replace with your Filestack policy

     const signature = 'd1db8c762144e5877e7e9efdd98994dfb1a5cc0c36cbe5c0d0f3c51d2d83e678'; // Replace with your Filestack signature

     const captionUrl = `https://cdn.filestackcontent.com/security=p:${policy},s:${signature}/caption/${handle}`;

     fetch(captionUrl)

       .then((res) => res.json())

       .then((data) => {

         setCaption(data.caption);

         setShowCaptionContainer(true);

       })

       .catch((error) => {

         console.error('Error generating caption:', error);

         setCaption('Error generating caption.');

       });

   };

generateCaption function is responsible for fetching a caption for the uploaded image.
policy and signature is used for security purposes to authenticate the request to Filestack.
captionUrl constructs the URL for the caption request using the file handle, policy, and signature.
fetch(captionUrl) sends an HTTP GET request to the `captionUrl` to retrieve the caption data.
setCaption updates the `caption` state with the fetched caption.
setShowCaptionContainer(true) makes the caption container visible.
catch handles any errors that occur during the fetch request.

7. copyCaption Function

   const copyCaption = () => {

     navigator.clipboard.writeText(caption).then(() => {

       setCopyMessageVisible(true);

       setTimeout(() => {

         setCopyMessageVisible(false);

       }, 2000);

     }).catch((error) => {

       console.error('Error copying caption:', error);

     });

   };

copyCaption function copies the generated caption to the clipboard.
navigator.clipboard.writeText(caption) copies the text contained in the `caption` state to the clipboard.
setCopyMessageVisible(true) shows the “caption copied” message.
setTimeout hides the “caption copied” message after 2 seconds.

8. JSX Render

return (

     <div className="app">

       <h1>Image Accessibility Viewer by Filestack</h1>

       <button className="upload-button" onClick={openPicker}>Upload Image</button>




       {showCaptionContainer && (

         <div id="captionDisplayContainer">

           <img id="uploadedImage" src={imageUrl} alt="Uploaded" />

           <p id="captionDisplay">{caption}</p>

           <button id="copyButton" onClick={copyCaption}>Copy Caption</button>

           {copyMessageVisible && <p id="copyMessage">Caption copied to clipboard!</p>}

         </div>

       )}

     </div>

   );

JSX is the component’s UI is defined using JSX syntax.
Upload Button is a button that triggers the `openPicker` function when clicked.
showCaptionContainer conditionally renders the caption display container if `showCaptionContainer` is true.
Uploaded Image displays the uploaded image using the `imageUrl` state.
Caption Display displays the generated caption using the `caption` state.
Copy Button allows the user to copy the caption by triggering the `copyCaption` function.
Copy Message conditionally displays a message indicating that the caption was copied to the clipboard.

9. Export Statement

   export default App;

export default App exports the `App` component as the default export, making it available for use in other parts of the application.

This component provides a user-friendly interface for uploading an image using Filestack, generating a caption for that image, and then allowing the user to copy the caption to the clipboard.

The next step is to add the CSS styles to your React App. You can get the CSS styles through the GitHub repository. Here is the explanation for the CSS styles:

CSS improves both visuals and the UX (user experience) itself. The body area has a light grey background with an image depicting scenery behind it. The image fills up the whole screen and creates a serene atmosphere. The font is clean Roboto, a modern sans-serif typeface, so this makes reading easier.A centered layout keeps content organized—nice division between elements.

Key Elements

This main heading, h1, is large enough and distinguishable from other texts due to the medium gray color tone used alongside bigger letters. The main heading (h1) is prominent. It has a larger font size and medium gray color, making it stand out. The text is dark grey, which gives enough contrast and readability.

Buttons are dynamic and interactive. They have a bright tomato red background with white lettering, creating a high level of contrast. Rounded edges add modernity to their look. Buttons feel responsive through hover and active states, with smooth transitions and slight motion on click.

The uploaded image shows rounded corners and a subtle shadow. This brings out the concept of material elevation, whereby the object seems to be lifted off the page. It’s responsive, meaning that it adjusts its size based on the container width so that it looks good on any screen.

A dialog box style was used for caption rendering. It has rounded edges, light gray bordering, and a white background. The area makes it clear where to focus attention while reading. A little shadow gives a slight illusion of elevation.

Users can use copy button to copy the generated caption.The button mimics the one used for uploading but is steel blue in color suggesting an alternate action.On copying the caption, there is a short-term validation message that appears briefly.This message does not interrupt user experience since it blends with the design yet remains conspicuous.

CSS transitions and transforms add interactivity. Hover or click buttons cause smooth transition in colors as well as positions giving some visual effects hence making them respond sensitively.These are part of what contributes to cleanest UX ever seen in reality .

Output

Open the terminal and run the command “npm start”. It will host your website at a local host of 3000. You can see the example given below:

You can see the website opened inside a browser.

Click on the upload image button in orange color. You will see the Filestack file picker as shown below:

Let’s suppose we upload an image given below:

You will see the image uploaded to the site with a caption at its bottom. There is also a button to copy the caption.

Conclusion

Your app must be available for all users. One of the main tools used to ensure this is image captioning. For example, Filestack’s Image Captioning automatically generates descriptive text for images to enhance accessibility for people with visual impairments. This ensures you meet legal standards and expands the audience for your app.

Integration of Filestack into a React JS environment makes adding this feature to an application very easy. This guide has shown how to make an application inclusive and user-friendly. Being accessible is not just a smart thing; it’s a must for any developer.

FAQs

What is the most reliable way to get image captions?

Use Filestack’s Image Captioning API for reliable captions.

What are the applications of image captioning?

Enhance accessibility, improve SEO, and support user engagement.

How much does Filestack cost for image captioning?

Filestack pricing varies by usage and plan. The basic plan starts at $69 per month.

Is Filestack secure for image captioning?

Filestack ensures security using encrypted connections, secure API keys, and customizable security policies. This protects your data during image processing and captioning.

Ayesha

Ayesha Zahra is a Geo Informatics Engineer with hands-on experience in web development (both frontend & backend). Also, she is a technical writer, a passionate programmer, and a video editor. She is always looking for opportunities to excel in her skills & build a strong career.