Home Digital Urdu: New Software Improves Data Analysis of Pakistan’s National Language

Digital Urdu: New Software Improves Data Analysis of Pakistan’s National Language

The extent to which social media sites like Twitter and Facebook play a role in the recent political uprisings in Tunisa, Egypt, Bahrain and so on continues to be a source of debate. What is more clear, however, is that the major languages of these regions are not well-served by electronic resources that make text analysis of these documents and data possible.

But now computer scientists have developed the first software system that will allow for the processing of documents in Urdu, the national language of Pakistan and one of the five most-spoken languages in the world.

The software will help lay the foundation for data mining in Urdu and provide for more accurate transliteration, as well as open the door for projects in similar languages. “This is the first comprehensive, natural language processing system for Urdu,” says Rohini Srihari, University of Buffalo associate professor of computer science and engineering.

The work is a joint project between her department and Janya, an Amherst-based company she founded that provides information extraction technology in multiple languages, including Chinese, Arabic, Pashto, and Russian.

Natural Language Processing and Urdu

The problem with data mining and sentiment analysis in other languages is that they don’t often have the same sort of “established electronic infrastructures” that we have for English and European languages. “If you are trying to do sentiment analysis – to find out what are the main topics people are talking about in a country, is there intensity building up over something and who is swaying opinion – then you must have an information extraction system,” Srihari says.

That is what she has been working on with researchers, something that can perform word segmentation (tagging parts of speech, for example) and entity-tagging (recognizing people, place, and organization names) in a raw, untranslated Urdu document.

“Voice of the Citizen” Through Social Media Data Mining

Srihari says she’s focused on the “voice of the citizen” in this project. “Some of the information is political and some of it is not,” she says, and despite the turmoil in the region, a lot of the social media chatter in Pakistan is about cricket.

Srihari presented her findings at a recent conference – “Blogs & Bullets: Social Media and the Struggle for Political Change” – at Stanford Universitiy. She says she became interested in Urdu because they were looking at blogs from different cultures. Noting that the advent of the Web has caused an explosion in online content in a variety of languages, Srihari says, “When you start looking at blogs in different cultures, you can really start to understand public sentiment and opinions.”

About ReadWrite’s Editorial Process

The ReadWrite Editorial policy involves closely monitoring the tech industry for major developments, new product launches, AI breakthroughs, video game releases and other newsworthy events. Editors assign relevant stories to staff writers or freelance contributors with expertise in each particular topic area. Before publication, articles go through a rigorous round of editing for accuracy, clarity, and to ensure adherence to ReadWrite's style guidelines.

Get the biggest tech headlines of the day delivered to your inbox

    By signing up, you agree to our Terms and Privacy Policy. Unsubscribe anytime.

    Tech News

    Explore the latest in tech with our Tech News. We cut through the noise for concise, relevant updates, keeping you informed about the rapidly evolving tech landscape with curated content that separates signal from noise.

    In-Depth Tech Stories

    Explore tech impact in In-Depth Stories. Narrative data journalism offers comprehensive analyses, revealing stories behind data. Understand industry trends for a deeper perspective on tech's intricate relationships with society.

    Expert Reviews

    Empower decisions with Expert Reviews, merging industry expertise and insightful analysis. Delve into tech intricacies, get the best deals, and stay ahead with our trustworthy guide to navigating the ever-changing tech market.