BytePlus Effects:

Elevating Customer Experience

with Immersive Video Filters
and AR Effects
Summary
This ebook offers an in-depth exploration of BytePlus Effects, a cutting-edge platform
specialising in immersive video effects, including stickers, filters, and augmented reality (AR)
effects. We delve into the platform’s precision-driven features, such as Detection & Tracking, and
the innovative applications of Generative Adversarial Networks (GANs). Through diverse industry
applications and insightful real-world case studies, BytePlus Effects showcases its capability to
enhance user engagement and transform digital strategies.
BytePlus Effects stands out with its continuous innovation, catering to the evolving needs of modern
businesses. The platform’s unique offerings are further enhanced by the BytePlus Creative Store,
which provides access to over 80,000 ready-to-use effects, ensuring a wide range of creative options
for businesses. Additionally, the BytePlus Creator Tool empowers users to craft bespoke video effects
and AR experiences, offering an intuitive interface for creating 2D, 3D, and interactive effects. This
combination of a comprehensive effects library and flexible creation tools positions BytePlus Effects
as a leader in the video effects SDK market.
For businesses eager to harness the power of a comprehensive video effects SDK, BytePlus
Effects offers unparalleled expertise and state-of-the-art tools. These tools can be integrated
swiftly, within as little as two weeks, enabling the creation of interactive experiences that
resonate with audiences and customers. Discover the transformative potential of BytePlus
Effects and how it can integrate the art of immersive video and augmented reality into your
business landscape.

The digital world is constantly evolving, and at This shift marks the beginning of a new
the heart of this transformation is advanced phase in how businesses interact with their
AR and visual effects, especially in the form customers, ushering in an era of transformation
of stickers and filters. These video effects and deeper engagement. By leveraging AR
are more than just digital embellishments; facial effects, stickers and filters, companies
they’re a blend of AI-driven technology and can engage with their audience in a more
creative expression that seamlessly integrates interactive and personalized manner. This
digital features with real-world imagery. This not only elevates user experience but also
innovative fusion offers businesses a dynamic fosters a deeper connection between brands
tool for user engagement. and their customers. It’s a move towards a
more immersive and responsive form of digital
For companies, AR effects, stickers and filters
interaction, where consumer engagement is not
are more than just a novelty; they’re a strategic
just seen or heard but vividly experienced.
asset. These tools empower your audience and
customers to express themselves creatively,
resulting in a surge of original, user-generated
content that amplifies a brand’s presence and
appeal. Additionally, for app developers, this
technology offers an avenue to incorporate
the latest AI advancements, ensuring that their
applications are not only engaging but also at
the forefront of technological innovation.

2
BytePlus: Redefining Digital Innovation and Growth

Emerging from the technological excellence of ByteDance, BytePlus is a game-changer

in the digital realm. It is dedicated to unlocking immense growth potential and amplifying
business success. With a specialised team of experts, BytePlus collaborates closely
with clients to enhance their products and user experiences, thereby driving substantial
growth. The essence of BytePlus lies in simplifying digital transformation, allowing brands
to deliver rich, data-driven experiences. This approach fosters a culture of innovation and
forward-thinking, essential in today’s fast-paced digital world.
BytePlus harnesses the power of ByteDance’s research and engineering prowess,
blending advanced technologies like machine learning and big data. This synergy
creates unique, engaging experiences that not only captivate your users but also drive
conversions. By continuously innovating customer experiences, BytePlus positions itself
as an indispensable partner for modern businesses seeking to thrive in the digital era.

BytePlus Effects: Crafting the Future of Augmented

Reality Apps

BytePlus Effects is revolutionizing app experiences with its innovative platform, offering
a vast array of immersive filters, stickers, and AR effects. Designed to enhance user
engagement, it features categories ranging from 2D and 3D to interactive, games, and
beauty, ensuring there’s something for every business need.
Central to BytePlus Effects is its advanced facial and body landmark tracking technology.
This ensures that effects are accurately and interactively placed, enhancing the realism
and engagement of AR experiences. The platform’s integration into existing apps
empowers businesses to stand out in the digital marketplace, capturing the imagination of
their audience.
The BytePlus Creative Store is a hub of inspiration, offering more than 80,000 ready-to-
use effects, constantly updated to drive higher engagement, conversion, and revenue
generation. This vast selection caters to a wide range of creative needs, allowing
businesses to tap into the latest trends in AR.
Further enhancing the platform’s capabilities, the BytePlus Creator Tool allows for the creation
of custom 2D, 3D, and interactive effects. With its user-friendly interface, businesses can
easily design unique visual experiences tailored to their specific requirements, making
BytePlus Effects not just a technology provider but a partner in innovation.
Through BytePlus Effects, businesses can explore the limitless possibilities of AR,
revolutionizing how they engage with their audience and customers.

3
BytePlus Effects: Detection and Tracking

Elevating Interactive The Detection & Tracking feature of BytePlus

Experiences with Precision- Effects is a foundational element of its AR
Driven Video Effects and technology suite. Expertly locating and tracking
AR Technology facial landmarks, this system ensures that AR
elements are precisely positioned and move in
sync with the user. It’s vital across various sectors,
Advanced Features particularly in entertainment with a focus on live
streaming, where dynamic filters, stickers, and AR
• Sophisticated face effects significantly enhance viewer experiences.
detection algorithms In social media, it enriches user interaction with
• Facial landmark tracking personalized AR elements; in the beauty industry,
using Convolutional Neural it allows for accurate virtual makeup trials; and
Networks (CNNs)
in e-commerce, it provides tailored virtual try-
• Real-time object rendering on experiences, enhancing online shopping.
and overlay Additionally, its impact is notable in the realm
Dynamic Applications of video content creation, offering innovative
possibilities that drive viewer engagement and
• Social Media: Enhances content popularity.
user engagement with
personalised AR effects for
captivating content. 1. Face Detection
• Short Video Generation:
Powers creative and BytePlus Effects utilizes advanced face detection
interactive video content with algorithms to accurately identify a user’s face
accurate face tracking and within an image or video frame. This technology is
dynamic overlays. crucial in live streaming and social media, enabling
• E-Commerce: Enhances a customized application of AR effects, filters, and
online shopping experiences, stickers, thereby enhancing each user’s digital
allowing customers to engagement uniquely.
interactively preview products
through AR technology.
2. Facial Landmark Detection
• Gaming: Creates immersive
gaming experiences by
accurately mirroring player Employing deep learning models, particularly
expressions and movements. Convolutional Neural Networks (CNNs), BytePlus
Effects precisely locates essential facial landmarks.
• Entertainment & Live
Streaming: Elevates live This capability ensures AR elements fit perfectly
streaming experiences in to each user’s face, a vital feature for personalizing
the entertainment industry, live streaming content and enhancing user
offering dynamic AR effects experience in AR applications.
for interactive and
engaging broadcasts.

4
3. Initial Calibration 6. Realistic Interaction

After landmark detection, the system calibrates This functionality allows users to interact with
to understand the face’s orientation, position, virtual objects through facial expressions or
and scale. This step is essential, particularly movements, particularly enhancing live streaming
for beauty apps and live streaming platforms, and social media experiences. Users can
where it enables realistic virtual makeup experiment with various dynamic effects, adding
applications and enhances live stream visuals. a layer of creativity and engagement to their
online presence.
4. Tracking and Overlay
7. Rendering and Display
Continuously tracking facial landmarks,
BytePlus Effects offers significant advantages The final step involves displaying the composited
in interactive gaming and live streaming. In image or video with virtual objects overlaid onto
gaming, it mirrors players’ expressions in real- the user’s face. This feature is critical in live
time, while in live streaming, it enriches the streaming and e-commerce, enhancing the viewer
broadcast with engaging AR effects. experience in streaming and providing a realistic
preview in online shopping.
5. Object Rendering
BytePlus Effects’ Detection & Tracking
In this phase, BytePlus Effects renders process showcases its commitment to
virtual objects such as 3D models, filters, or precision and user engagement, making
masks onto the user’s face. This feature is
it an indispensable tool for businesses
invaluable in live streaming and social media,
across various industries, especially in
where it brings characters and effects to life,
transforming standard content into engaging entertainment with a strong emphasis on
experiences for audiences and creators. live streaming.

106 Keypoints of the Face 280 Keypoints of the Face: More keypoints of the lips, irises, and eyebrows
are added to the 106 keypoints.

Beyond the precision of detection and tracking, BytePlus also delves into the realm of AI-driven
creativity. This brings us to the innovative use of GANs in video effects, a field where BytePlus is
setting new standards.

5
BytePlus Effects: Generative Adversarial
Networks (GANs)
GANs stand as a cornerstone in the field of The training process of GANs is marked by
artificial intelligence, particularly in machine a competitive dynamic between these two
learning and deep learning. These networks networks. This competition drives both to
are designed to generate data that closely improve, leading to highly realistic outputs.
mimics real-world inputs. This capability The Generator constantly aims to produce
is especially valuable in video effects data that is indistinguishable from real
applications, where GANs can be used to data, effectively ‘tricking’ the Discriminator.
create a wide range of effects, from realistic Conversely, the Discriminator works to
stylisations, like comic or painting styles, to become more adept at distinguishing fake
more complex data generation tasks. data from real. This adversarial process
is iterative, with each network striving to
GANs have two critical components: the
outperform the other. Over time, this results
Generator and the Discriminator. Imagine
in the generated data becoming virtually
them as two artists in a contest: one creates
indistinguishable from actual, real-world data.
paintings (the “Generator”), and the other
This technology allows for creating effects
judges them (the “Discriminator”). The
that range from lifelike renderings to artistic
Generator creates images, audio, or text
transformations, enhancing the immersive
from random noise, striving to make them as
experience of AR applications.
realistic and as close to real-world data as
possible. This could be anything from images
and audio to text. As the training progresses,
this network refines its ability to create
data that is increasingly realistic, effectively
blurring the lines between generated and
authentic data.
In contrast, the Discriminator network has a
different role. Trained with real-world data, it
tries to distinguish whether the Generator’s
creations are real or fake. So, it receives
both real data, such as genuine images, and
the generated data from the Generator and
tries to tell them apart. Essentially acting as
a sophisticated classifier, the Discriminator
assesses each data point to determine
whether it’s a genuine article or a fabrication
by the Generator.

6
Applications of GANs That Make AR
More Immersive and Interactive

Applications of GANs in AR GANs play a transformative role in augmenting the

capabilities of video effects offering a breadth of
applications that significantly enhance both the
Realistic Object Rendering realism and interactivity of AR experiences. Here
are some pivotal ways GANs contribute to AR:
• Generates lifelike 3D models
for AR, merging virtual and real • Realistic Object Rendering: Employing GANs to
worlds seamlessly. create detailed 3D models offers a significant
Image Augmentation leap in AR realism. These models, when
integrated into AR environments, form lifelike
• Enhances AR visuals with virtual objects that blend seamlessly with
super-resolution and creative
the real world. This integration heightens the
style transfers.
authenticity of AR experiences, making them
Content Generation more convincing and immersive.
• Creates interactive, synthetic • Image Augmentation: In AR, GANs excel in
characters and environments refining real-world images and video frames.
for immersive AR. They are instrumental in tasks like image
Object Recognition and Tracking super-resolution and style transfer, not only
improving visual quality but also introducing
• Improves accuracy in identifying
creative transformations. This application
and following objects in AR.
ensures that AR visuals are not just clearer but
Environmental Augmentation also more engaging and aesthetically varied.
• Seamlessly integrates virtual • Content Generation: GANs facilitate the
elements into real-world creation of dynamic synthetic content, including
settings via AR. unique characters, avatars, or backgrounds.
Style Transfer and Effects These elements, responsive to user interactions,
inject a level of creativity and depth into AR
• Transforms real-world views
applications, fostering an enriched, engaging
with artistic styles in real-time.
user experience.
Data Augmentation

• Generates training data to refine

AR systems, enhancing object
recognition and tracking.

These applications of GANs

demonstrate their critical role
in enhancing the realism and
creativity of AR experiences,
contributing to more immersive
and interactive user interfaces.

7
• Object Recognition and Tracking: • Style Transfer and Effects: Real-time style
Another significant application of GANs transformation of the environment, enabled
is enhancing the precision of object by GANs, introduces a creative twist to
recognition and tracking within AR AR. Users can witness their surroundings
systems. Trained on diverse real-world morphed by various artistic styles or
objects, these networks enable AR visual effects, adding a layer of artistic
platforms to more accurately identify and engagement to the AR experience.
follow objects, ensuring more stable and
• Data Augmentation: Crucial for refining AR
reliable interactions in the AR space.
technologies, GANs can generate additional
• Environmental Augmentation: GANs can data to train AR systems, particularly in
craft virtual elements that integrate smoothly improving object recognition and tracking.
into physical settings via AR devices. This This function is key to enhancing the
feature can revolutionise interior design, accuracy and efficiency of AR applications,
education, or entertainment, allowing for ensuring they respond more intuitively to the
the virtual alteration of real-world spaces in real world.
inventive and functional ways.

In conclusion, GANs stand as a cornerstone in video effects, expanding the scope of what’s
possible in augmented reality, from rendering realistic virtual objects to transforming the
aesthetics of real-world environments. As these technologies continue to evolve, they promise
to make virtual experiences not only more immersive but also more interactive and user-friendly,
shaping the future of our digital interactions with our audience and customers.

8
Rendering and Driving:

Powering
Interactive
Experiences
with BytePlus
Effects

Complementing the rendering engine is the sophisticated driving technology. BytePlus

Effects’ assets intelligently follow and mimic facial expressions and body movements.
This synchronisation between the user’s real-world actions and the video effects assets
creates a highly interactive and immersive experience, enhancing communication
platforms, fitness, and gaming applications. High-quality rendering makes digital elements
life-like, and essential for user engagement. Meanwhile, the driving technology ensures
dynamic interaction, key to intuitive user experiences. Together, they transform video
filters and AR effects from digital overlays to convincing, interactive experiences that blur
the digital and physical world boundaries.
The significance of rendering and driving cannot be overstated, as they are pivotal in
creating a powerful and compelling user experience. High-quality rendering brings digital
elements to life with stunning realism, making them virtually indistinguishable from the
real environment. This level of detail is essential in capturing and retaining user interest.
Meanwhile, the driving technology ensures these rendered elements are not static but
dynamically interactive, responding to users’ movements and expressions. This interaction
is key to making user experiences feel intuitive and natural, fostering a deeper sense of
immersion and engagement.

9
Diverse AR Innovations: The Spectrum
of BytePlus Effects

Exploring BytePlus Effects BytePlus Effects is revolutionising business

and AR Applications interactions with its advanced suite of video
effects and AR filters, each uniquely designed
to cater to a myriad of industry-specific needs.
BytePlus Effects is revolutionising This platform is not just an array of digital tools;
business interactions with its it’s a transformative force in hyper-immersive
advanced suite of special video
video stickers and filters, bringing unparalleled
effects technologies, each uniquely
designed to cater to a myriad interactivity and engagement to various sectors,
of industry-specific needs. This from gaming to e-commerce to beauty.
platform is not just an array of
BytePlus Effects offers innovative solutions
digital tools; it’s a transformative
force in Augmented Reality, to enhance your audience and customer
bringing unparalleled interactivity experiences. Each feature, be it Body Motion,
and engagement to various sectors. Gesture Recognition, Beauty, Portrait, or Sticker
Effects, is crafted to elevate how businesses
Body Motion connect with their audience, merging the digital
and physical worlds in an unprecedented manner.
• Real-time human body
motion detection
• Has gaming and sports
1. Body Motion
Gesture Recognition Imagine a world where technology intuitively
understands human movement. BytePlus
• Advanced hand gesture tracking
Body Motion does just that. It’s not just about
• Enhances user-device interaction detecting people in images or videos; it’s about
Beauty understanding the nuances of human motion in
real time.
• AR beauty and makeup effects
• Ideal for photography, live • Product Features
streaming
- Real-time detection of human bodies
Portrait
- Multi-person detection
• Detailed facial
- Half-body detection
recognition technology
• Powers interactive stickers, - Bounding box detection
AR makeup - Generation of keypoints
Sticker Effects - 18 keypoint detection including head,
• Extensive sticker library shoulders, feet and more
for creativity
• Used in content
creation, education

10
• Application Scenarios 3. Beauty
- Interactive Games: Enhances gaming
experiences by accurately detecting and BytePlus Beauty seamlessly blends the
replicating human postures and actions transformative power of AR with the world
in real-time, essential for motion-sensing of beauty, redefining the aesthetics of digital
games, creating a more immersive and imagery with a sophisticated touch. It enhances
interactive gameplay environment. personal appearance in photos and videos with
a touch of AR magic.
- Sports and Fitness: Offers real-time body
tracking, allowing sports and fitness apps • Product Features
to analyse and score physical actions,
- Special effects and makeup filters
aiding in performance improvement and
personalised training programs. - Face and body morphing

- Content Creation for Social Media: - Real-time effects in videos and photos
Empowers creators in social media and - Multiple adjustable portrait effects
short video platforms by enabling dynamic
body movement tracking, facilitating more - Skin retouching and facial morphing filters
engaging and interactive content creation. - Diverse makeup and AR micro-shaping
effects such as lipstick, blush, eyeshadow,
2. Gesture Recognition hair dyeing and more
- Nearly 100 diverse portrait filter templates
Gesture Recognition is BytePlus’s way of
turning subtle hand movements into powerful • Application Scenarios
commands. It’s about creating a seamless
- Photos and Videos: Revolutionises digital
interaction between humans and machines.
photography and videography by allowing
• Product Features users to apply various beautifying effects
in photo-taking, video recording, and
- Single-hand, two-hand, and multi-person editing scenarios, enhancing both personal
gesture recognition and professional content.
- Bounding box detection - Live Videos: Elevates live streaming and
- Keypoint detection video calls by providing real-time beauty
effects, ensuring users look their best in
- 22 hand keypoints tracking including
every frame, crucial for broadcasters and
fingertips, joints, palms
influencers in maintaining a professional
- Recognises 4 types of hand gestures and appealing on-screen presence.
(static and dynamic)

• Application Scenarios

- Interactive Stickers: Enhances user

interaction in digital platforms by enabling
the control of special effects through
intuitive hand gestures, making
experiences fun and engaging.
- Gesture-Based Control: Transforms how
users interact with smart devices, allowing
natural and intuitive control through
various hand gestures, which can be
integrated into smart home systems and
interactive applications.

11
4. Portrait 5. Sticker Effects

BytePlus Portrait is a groundbreaking SDK BytePlus Sticker Effects infuses digital content
that brings unparalleled depth to facial with a lively and creative spark, offering a vast
recognition technology, capturing every array of stickers for every imaginable scenario.
nuance of human expressions.
• Product Features
• Product Features
- Thousands of stickers with realistic effects
- Detection of up to 280 keypoints in portraits
- Covers 2D, 3D, makeup, condition-
- Real-time multiple face detection in triggered, and virtual IP effects
images and videos
- Continuously updated sticker library
- Recognition of six key facial movements
- Sticker store for the latest trends
- Identification of facial attributes, including
- Extensive resources for
emotions and gestures
sticker customisation
• Application Scenarios
- In-house sticker-making tool
- Interactive Stickers: Enables users to
• Application Scenarios
activate and control special effects
through natural face movements, - Beauty Tools: Enhances photo and
enhancing interactive digital content. video editing by adding artistic flair and
- AR Makeup: Offers a virtual try-on personalisation, ideal for content creators
experience for makeup or contact lenses, and beauty enthusiasts.
leveraging precise keypoint detection for - Live Streaming: Offers real-time effects
a highly realistic and personalised preview. that add visual interest and professionalism
to live streams and video calls, crucial for
influencers and broadcasters.
- Online Education: Makes learning more
engaging and interactive with stickers
relevant to lesson content, enhancing the
virtual classroom experience.

As we have explored the dynamic capabilities of BytePlus Effects,

it becomes evident that this platform is not just a technological
advancement but a key to unlocking new dimensions in customer
engagement and business innovation. The diverse range of video
stickers, filters and AR effects – from body motion and gesture
recognition to beauty, portrait, and sticker effects demonstrate
BytePlus’ commitment to empowering businesses across sectors.

12
Case by Case: BytePlus Effects Driving Change

Karasta
To truly appreciate the impact of BytePlus Effects, let’s delve into real-world applications
to see how brands have harnessed BytePlus Effects’ technology for remarkable results.

Karasta, a social karaoke platform, leveraged BytePlus Effects to enhance user

engagement and stand out in a competitive digital landscape. Integrating state-of-the-
art AR effects, including beautification and micro-shaping, Karasta transformed standard
karaoke sessions into captivating visual spectacles. This strategic integration led to a
significant increase in user engagement, exemplified by a 15% increase in daily active
users and a 20% rise in user dwell time.
BytePlus Effects has proven its capability to revolutionise user engagement across various
industries. Its innovative approach, continuous development, and proven effectiveness
make it an ideal choice for businesses aiming to leverage AR technology. Whether
enhancing customer experiences, boosting user engagement, or driving business growth,
BytePlus Effects provides a robust and reliable AR solution.
Explore the complete Karasta case study and learn more about their journey with BytePlus
Effects on our website.

13
Selecting Excellence: Why BytePlus Effects
is the Industry Benchmark
BytePlus Effects distinguishes itself as a leader in video effects, offering unparalleled precision,
performance, and innovation. Its strengths lie not only in its advanced technological foundation
but also in its commitment to continuous evolution and responsiveness to business needs.

1. Leading Technology 3. Business-Driven Optimisation

BytePlus Effects stands out for its cutting-edge BytePlus Effects is continually refined
algorithms, which ensure exceptional precision in response to the real-world needs of
and performance. This technological excellence businesses. This approach ensures that the
translates into realistic and seamlessly textured platform not only meets the current demands
video effects, leveraging precise keypoint of various industries but also anticipates future
technology. The platform’s ability to render trends, enabling businesses to stay ahead of
complex textures, from metal to fabric, with the curve.
natural finesse, sets it apart in the special video
effects and AR landscape. Its sophisticated 4. Tried and Tested
rendering and driving technologies, for instance,
enable lifelike replication of human movements The platform’s reliability and effectiveness
and expressions, making video filters and AR are proven, with a global user base in the
effects incredibly immersive and realistic. billions. This widespread adoption across
different sectors underlines its robustness
2. Continuous Upgrades and versatility, making it a trusted choice for
businesses worldwide.
BytePlus’s commitment to continuous
improvement is a testament to its dedication
to staying at the forefront of video effects and
AR technology. The platform undergoes regular
updates, ensuring it remains aligned with the
latest advancements in the field. This relentless
pursuit of excellence means businesses always
have access to the most advanced special
video effects and tools.

14
BytePlus Effects epitomises the fusion of If you’re intrigued by the potential of BytePlus
technological sophistication with practical Effects to transform and elevate your business,
application, marking a new era in video effects. we’re here to assist. Our team is ready to guide
This ebook has traversed the landscape of you through the possibilities and practical
BytePlus Effects’ offerings, from groundbreaking applications of our video effects and AR filters
detection & tracking to the innovative use of that can be integrated in as short as two weeks.
GANs in AR, showcasing how these technologies
are not mere digital tools but transformative For a deeper understanding
forces reshaping business interactions. of how we can be part of your
BytePlus’ commitment to precision, innovation, digital strategy and to explore a
and responsiveness to business needs shines partnership, feel free to reach out
through in its broad spectrum of video effects
to our specialists about BytePlus
and AR technologies. The BytePlus Creative
Store enriches this offering with an extensive Effects and consider how the art of
collection of over 80,000 ready-to-use effects, immersive video and augmented
catering to diverse business requirements. reality can be integrated into your
Complementing this, the BytePlus Creator
Tool empowers businesses to craft unique
business landscape.
and tailored AR experiences with an intuitive
interface for creating 2D, 3D, and interactive
effects. These innovative tools not only
enhance user experiences but also redefine
how businesses engage and connect in an
increasingly digital world.

Ready to Transform Your Vision into Reality?

Dive into the realm of BytePlus Effects and take your app
and content creation endeavors to new heights today.
Experience the magic first-hand with our demo effects!
For an in-depth consultation with a BytePlus solutions
specialist, connect with us!

15