We need your support. Know More

How Content Moderation Is Failing as Kids Lap up Culture of ‘Gangster’ Violence Online

Alishan Jafri and Vipul Kumar
15 hours ago
Blanket bans, lacking contextual consideration, infringe on freedom of speech and journalistic work. On the other hand, gigantic loopholes ensure that a flourishing culture of violence is accessible to minors.

Eleven-year-old Raman has a life plan in place – he will get married when he turns 12 next year and to sustain his family, he will “become Lawrence Bishnoi.”

Raman (name changed) is the cousin of Vipul Kumar, one of the two authors of the piece. Raman’s father, a skilled migrant worker, moved the family to Gujarat from their native village in Bihar a year ago. The move left Raman without friends. His phone was his only source of entertainment and Bishnoi, his new hero. What did Raman know about Bishnoi? 

“Lawrence wanted to kill a fox as a child, but Salman Khan stopped him, so now he wants to kill Salman,” Raman said, unfaltering in confidence.

Though wildly inaccurate, his fascination with Bishnoi was steady. Bishnoi is in jail in Gujarat but has now found renewed space in front pages since he was named by the Canadian government as someone whose gang was allegedly working with the Indian government to foment violence in Canada. Then there are his multiple threats to the actor Salman Khan. Through all of this, Bishnoi has become a hero to young men and children, primarily thanks to social media.

Bishnoi himself does not have a personal social media account. His name has been banned from yielding search results on Meta. And yet, videos on him, posted by fawning social media participants, go viral all the time. As per the National Investigation Agency and various state police forces, content featuring Lawrence Bishnoi’s name alone has an impact on youngsters. Certain groups attempt to sell illegal firearms in his name, while others sell anything that passes for “Lawrence Bishnoi merchandise”. 

Recently, The Wire reported on e-shopping platforms and websites like Meesho selling the latter. It caught the attention of other news platforms and caused massive online uproar. Soon, these platforms took down their products. 


On November 8, the Maharashtra cyber police filed a first information report against sellers and platforms under sections 192, 196, 353, and 3 of the Bharatiya Nyay Sanhita, and section 67 of the Information Technology Act, over listing products that glorify criminal figures. A press note by the police said that idolising criminals through merchandise can potentially “encourage impressionable youth to admire and imitate unlawful behaviour.” 

However, gangster t-shirts are a small cog in the wheel that normalises gun violence, glorifies gang crimes and makes “becoming Lawrence Bishnoi” a viable career option for children like Raman.

Music

While gangster-inspired pop music is not new in the world or in India, social media has taken it to hitherto unseen levels of popularity. Lawrence Bishnoi-themed songs flood social media and are on major streaming and music hosting platforms like Apple music, Jio Savan, Amazon Music, Soundcloud, and Spotify. Not only do these songs explicitly glorify gang crime but they also feature gangsters by name, in some cases featuring clips of their audio threats and their photographs in posters. Some of these songs are fan tributes to Bishnoi actions; others declare support for him.

‘Fan chhiye Lawrence Bishnoi ke’

This song by Suman Sona and Gaurav Thakur was uploaded on Amazon Prime Music on October 25, 2024. It is also available on Jio Saavn and Soundcloud. The  song has been uploaded on a YouTube channel with over 663,000 subscribers.

A part of the Bhojpuri-Maithili lyrics in English say, “Don’t show your power because I am a big fan of Lawrence Bishnoi… Don’t act like a strongman, otherwise we’ll treat you in a way that nobody will be able to find your dead body.”  

‘Lawrence Bishnoi-Titu Remix’

Four days later, on October 29, a similar song was released on Apple music. The singer on the poster – which also has Bishnoi’s photo – poses with a pistol.

The lyrics say: ‘The Hindu lion [presumably Bishnoi] has arrived. He kills his opponents with a roar. If someone puts a finger, he will tear them apart.’

‘LBG (Lawrence Bishnoi Gangster)’: Samrat Shooter Sam Roy

Available on Jio Saavn, Apple Music, Spotify, and Amazon Prime, this song has received 1.8 million plays on just YouTube. Its various versions have lakhs of views. The song starts with Bishnoi’s voice – his audio threat from his interview to ABP News after the murder of singer Sidhu Moosewala. “No matter how big or powerful the person is we will kill him (Koi bhi kitna bhi bada tagda aadmi ho mashhoor ho, ham usko maar denge),” Bishnoi’s processed voice says.

 

 

Is there legal framework to regulate such songs? Are these songs illegal? Or will banning them be illegal?

Apar Gupta, lawyer and co-founder of the Internet Freedom foundation feels that these songs need to be viewed within the framework of existing content control regulation, to which end there are various and substantive provisions of the law. “It is also important to consider that users who post this kind of content not only come under the framework of laws but also under user agreements of platforms. For instance if I want to publish on Spotify or Youtube, there are certain content restriction guidelines which apply to specified categories such as children,” he said.

According to Gupta, “There are some user agreements made by the platforms themselves. While the IT rules say that the platforms have an obligation of enforcing their end user agreement with users. Ideally, if those conditions are not satisfied, any user can file a complaint with the platform. And If the platform doesn’t act then you can approach the grievance appellate committee.”

But what of concerns of censorship? “Content in this particular category may be socially undesirable and potentially may have a tendency of escalating aggression in society but it is important to bear in mind what are the other ways to tackle it rather than just banning content,” he said. 

We reached out to the above-mentioned streaming platforms to inquire about their moderation of gang-related content, including measures to protect minors, and the alignment of this content with their community guidelines. 

Amazon and Youtube informed The Wire that the content in question has been removed due to non-compliance with their guidelines. The next day, Amazon took down almost all such songs related to Bishnoi, erasing entire playlists. YouTube only acted against the three sample links shared by The Wire. However, these very songs are available in different iterations on YouTube, especially as  short form content.

Which brings us to the second chapter of this social phenomenon…

The Reels and Shorts revolution

What compounds the reach of such songs and adulatory content is the viral short form. There are hundreds and thousands of Reels (short videos on Instagram) with millions of views. And Bishnoi far from the only gangster being glorified in them.

One can find similar tracks on the 20-year-old slain Ujjain gangster Durlabh Kashyap. His posthumous fandom is such that teenagers dress up like him, get his images tattooed on their bodies, and even commit murders in his name. Songs and music videos made to glorify him show young teenagers dressing up as Durlabh and engaging in loot, dacoity, and violence – all on Reels. 

Some viral Reels also promote Dawood Ibrahim, the famed lynchpin of Mumbai’s underworld. One Reel trend promotes Dawood as “India’s top badshah (king).” Against the audio backdrop of film dialogues, these Reels show the makers of the Reel as subsequent “kings” after Dawood Ibrahim and gangsters Salman Lala or Maya Bhai. 

One such Reel has over a crore and 20 lakh views and has been shared 2.5 lakh times. It shows footage of Dawood with dialogue from the film Shootout at Vadala – “Badshah ki gali mein aakar uska pata nahi poochte, gulamo ke jhuke hue sar khud ba khud uska pata bata dete hain”. Translated, it means, “You don’t ask for a king’s address. The bowed heads of his slaves lead you to him.”

Salman Lala, a young gangster from Indore, is another favourite. In many of these being jailed has been glorified as sacred duty.

In another viral trend, one ‘Addu Defaulter’ is presented as the next big gangster. There are multiple videos showing him during his court visits and smoking in prison. Iterations of Reels fawning over Addu Defaulter are shared multiple times.

A single audio clip – like the one below which in one version features the voice of a gangster Billu Sanda – has been used for millions of Reels.

Some teenagers add “Defaulter” to their Instagram handles and claim that they have committed murder. The number 302 – the section for murder under the Indian Penal Code – is an indicator too. They also post pictures in handcuffs or with policemen, to claim that they are members of Defaulter’s group.

Instagram account of a teenager mentioning a murder charge in his bio.

Many of these Reels clearly feature minors. One shows a possible kindergartener packing bullets in her bag. The Wire’s video analysis had showed how multiple men and teenagers committed gun crimes after having been inspired by some of these gangsters and gang content.

‘Jab 16 Saal Ka Tha Khalnayak Dekh Li’ on JioSaavn with over 990,000 listens.

We also found that on YouTube, much of this content is readily available. To check this, we created a sample teenager account with the alias Rahul Kumar (13 years and 7 months old). We searched for the above-mentioned songs and content. All were available to the minor’s account. We changed Rahul’s age to 11 and then the ‘desired content’ setting to that of ‘9+ years old’. The content was still readily available. Soon, YouTube began suggesting more of it. On YouTube, you can control the content a child consumes till they reach the age of 13, while on Meta, the minimum age for creating an account is 13. 

Sources at YouTube told The Wire that their policies strictly prohibit content that supports violent extremist or criminal organisations, including material that promotes violence or gang activity. After reviewing flagged videos from The Wire, YouTube said that they removed them in compliance with these guidelines. However, such content is still accessible and suggested to users under 18, even for those who have set the desired content setting to 9+ years of age. YouTube says that for users under 18, it applies added protections by defaulting uploads to private, enabling wellbeing reminders, turning autoplay off, blocking targeting through personalised ads, and limiting repetitive recommendations on sensitive content like non-contact fights. However, as we saw, sensitive content, especially content inciting or glorifying gang violence or extremism, is quite easily accessible and may be recommended even to users under 18.

YouTube says that they review flagged videos individually, removing those that breach guidelines and applying age restrictions on content that may be inappropriate for viewers under 18. In cases of graphic content, warnings and age-gates ensure only users aged 18 and over can view it. Throughout our search as 11-year-old Rahul Kumar, we didn’t find age restrictions on any of these channels, Shorts, or videos.

We conducted a similar experiment for Meta. We created an Instagram account for a 13-year and seven-month-old boy named Rahul Kumar. We searched with ‘Lawrence Bishnoi’. Initially, Instagram did not show much content. But soon the feed was filled with violent content related to Bishnoi and other gangsters. Engaging with a single Reel brought a cascade of similar, violent content into “13-year-old Rahul Kumar’s” feed.

After being shown Reels related to gang culture, the algorithm automatically began recommending content supporting violence for cow vigilantism, pseudoscience, targeting Muslims, and promoting the conspiracy theory of “love jihad”. One Reel featured visuals of a cow in tears, and the lyrics said, “We will put brass [bullets] in your chest if you kill cows.” Another suggested that Muslim men murder Hindu women.

Since our focus was solely on gang content, we did not search for any of this – it was automatically recommended. 

Jonathan C. Ong, professor and director of Glotech Lab and a scholar on misinformation pointed out that subcultures are dynamic but not exactly welcoming at the outset.

“What an outsider might see as a passing comment could actually be a dog whistle or an invitation to join a radical group. This is why both human moderators and AI moderation systems struggle to detect such content,” he said.

‘Content moderation’

Meta’s transparency page says that there is an “increase in content actioned for Dangerous Organizations and Individuals on Facebook and Instagram” due to continued updates made to their “proactive detection technology”.

Despite these tall claims, a cursory search would show you that much of such content is readily available even when Meta has categorised Lawrence as a “dangerous individual” in 2024. Many others like Kashyap and Dawood are also “search banned”.

Search results in Hindi on Facebook.

While searching for Lawrence Bishnoi on Instagram and Facebook, the initial search yields no result, followed by a pop-up warning that the term may be associated with dangerous organisations and individuals. However, this warning appears only for searches in Roman script and Punjabi (Gurmukhi script on Instagram), but not in Devanagari, which is used for Hindi and some other regional languages in India. This indicates that the moderation is primarily aimed at English-speaking users. Various research studies claim that Meta lacks proper training for the platform in the Hindi or the  Indian context. Sources close to Meta told the Wire that  search for “Lawrence Bishnoi Gang” in Hindi is also censored. However, when we searched for “लॉरेंस बिश्नोई गैंग” what appeared on our screens are different variations with changes. 

The inability to properly moderate content in non-English languages raises concerns about content moderation inequality. On the other hand, existing moderating strategies may be counterproductive and inspire social media users to be creative with their searches. Whistleblower Francis Haugen told The Guardian in 2021 that she believes Facebook’s “prioritisation of profits over people” endangered the lives of people, especially in the Global South, a charge that Facebook had denied. She has also said that 87% of the spending on combating misinformation at Facebook is spent on English content when only 9% of users are English speakers.

A recent report, published by the Washington based think-tank Center for the Study of Organized Hate (CSOH), details how the platform amplifies and promotes violent and hateful content, allowing accounts that post hate crimes to fundraise on Instagram in violation of Meta’s content policies. 

Jonathan also feels that content moderation inequality may be contributing to the proliferation of gangster-related content for young people in India. “We know that these platforms often invest less in monitoring content in non-English languages, especially in the Global South. When it comes to smaller platforms unlike Meta, there is even less investment in trust and safety,” he said.

Even as Meta claims to have a proactive approach in combating promotion of dangerous activity on its platforms, we found that its techniques are largely ineffective in the case of gangsters like Bishnoi, and yet have an overtone of censorship.

Recently, we posted a story on Instagram asking, “Has Meta banned the word ‘Lawrence Bishnoi’?” The story was deleted with a notification saying, “It looks like you shared or sent symbols, praise or support of people and organizations that we define as dangerous…” 

Similarly, journalist Tanushree Pandey shared on Instagram an interview she conducted with the son of Baba Siddique, who was allegedly killed by Lawrence Bishnoi’s shooters. Her post was removed just minutes after it was published, without regard for context. In another instance, The Quint journalist Himanshi Dahiya shared our aforementioned video: “How gangsters became cool…”. Her caption mentioned Lawrence Bishnoi and was taken down by Instagram. Such blanket bans, lacking contextual consideration, infringe on freedom of speech and journalistic work. 

Make a contribution to Independent Journalism