Integrate Agora Extension into your Appbuilder app

This guide walks you through embedding interactive features from the Agora Extension Marketplace into your app. Agora's Extension Marketplace allows you to quickly add first- and third-party features like noise canceling and face/voice filters to your Voice Calling, Video Calling, Broadcast Streaming, and Interactive Live Streaming apps with minimal effort.

Installing the Extension

Visit the Agora Extension Marketplace and choose a video extension to integrate into your app.

For example, to install the Super Clarity Extension, run the following command:

npm i agora-extension-super-clarity

Integrating the Extension

If you haven't already, create a PreferenceWrapper.tsx file. This file wraps the application logic for the Precall and Call screens, where you'll integrate the Agora extensions.

Step 1: Import Built-in Extensions

import { useVirtualBackground, useBeautyEffects } from "@appbuilder/react";
const { virtualBackgroundProcessor } = useVirtualBackground();
const { beautyProcessor } = useBeautyEffects();

Step 2: Register the New Extension

Next, register your new extension with the Video SDK by adding the following code to your PreferenceWrapper:

import { SuperClarityExtension } from "agora-extension-super-clarity";
const [superClarityProcessor, setSuperClarityProcessor] = useState<ReturnType<
  SuperClarityExtension["_createProcessor"]
> | null>(null);

const superClarityExtension = new SuperClarityExtension();
window?.AgoraRTC.registerExtensions([superClarityExtension]);
setSuperClarityProcessor(superClarityExtension.createProcessor());

Step 3: Integrate the New Extension into the Video Processing Pipeline

Agora's Web SDK processes video through a structured pipeline, which consists of several stages: capture, preprocessing, encoding, transmitting, decoding, post-processing, and playback. Each stage plays a crucial role in delivering a high-quality video experience.

The preprocessing stage is where most extensions, including virtual backgrounds, beauty effects, and custom enhancements like Super Clarity, are applied. In this stage, the video data is modified or augmented before it is encoded and transmitted. This allows you to layer multiple effects and optimizations onto the video stream.

To integrate your new extension into this pipeline, you can chain it with other extensions, ensuring that the video is processed through each stage sequentially. Here's an example of how to do this:

import { useRtc } from "@appbuilder/react";
const { RtcEngineUnsafe } = useRtc();
const localVideoTrack = RtcEngineUnsafe?.localStream?.video;

React.useEffect(() => {
  if (
    localVideoTrack &&
    virtualBackgroundProcessor &&
    beautyProcessor &&
    superClarityProcessor
  ) {
    localVideoTrack
      ?.pipe(virtualBackgroundProcessor)
      ?.pipe(beautyProcessor)
      ?.pipe(superClarityProcessor)
      ?.pipe(localVideoTrack?.processorDestination);
  }
}, [localVideoTrack]);

Step 4: Pass the New Extension Instance

To use the new extension, pass its instance through the PreferenceWrapperContext.Provider:

<PreferenceWrapperContext.Provider value={{ superClarityProcessor }}>
  {props.children}
</PreferenceWrapperContext.Provider>

Step 5: Disable Registration of Built-in Extensions

Since all extensions are already registered in the PreferenceWrapper, disable their registration in the embedded SDK to avoid redundancy as below :

AgoraAppBuilder.joinPrecall(props?.meeting, "", isBot, {
  disableVideoProcessors: true,
});

Using the Extension

To utilize your registered extensions, retrieve the processor instance from the PreferenceWrapper Provider. Ex:

import { usePreference } from "../PreferenceWrapper";
const { superClarityProcessor } = usePreference();

// Add the Super Clarity effect to the video stream
superClarityProcessor?.enable();

// Remove the Super Clarity effect from the video stream
superClarityProcessor?.disable();