Last September, when Typhoon Sanba smashed into the Korean peninsula, it packed winds so strong that they sent rocks flying through the air like missiles and caused massive power outages. “Hwangsa” storms, carrying dense clouds of yellow dust from China’s Gobi Desert that are sometimes loaded with heavy metals and carcinogens, sweep across the peninsula from West to East.

9.3 Petabytes Of Storage For The KMA

Menaced by such destructive weather phenomena, South Korea is upgrading its national weather information system with the goal of understanding weather patterns better and predicting better the location and ferocity of weather events. The upgrade being installed by the Korean Meteorological Administration increases the agency’s data storage capacity by nearly 1,000% to 9.3 petabytes, making it Korea’s most capable storage system.

The KMA project dramatically illustrates today’s big data phenomenon and its impact on weather forecasting. Thanks to the rapid spread of sensors and satellites, and to the increase in computer number-crunching speeds, it’s possible to forecast weather changes more accurately and with improved detail – potentially saving thousands of lives and safeguarding property. 

Increasing evidence of climate change worldwide is prompting governments and scientists to take action to protect people and property from its effects. But to take effective action, they need to know understand a lot more about the weather – everything from what’s going to happen tomorrow to what’s coming next year. For instance, leaders of the city of Hoboken, N.J., in the United States, which flooded badly last fall during Hurricane Sandy, are considering building a wall around Hoboken to keep the tidal Hudson River at bay. The problem is, if they don’t build high enough the wall could end up turning the city into a giant bathtub rather than keeping rising waters out.

Listen To Deep Thunder

IBM Research scientists are working to bring the most sophisticated data analytics to bear on weather forecasting. Their long-term weather analysis project, called Deep Thunder, combines data with sophisticated mathematical algorithms and computing power.

The scientists established at test bed in the New York City metropolitan area, where they set up a three-dimensional grid of thousands of blocks. That makes it possible for them to run calculations that produce very precise weather forecasts for a particular locale. Using this capability, the team was able to predict with remarkable accuracy the snowfall totals in New York City during the mammoth snow storm that blanked the northeastern United States in February – and also to predict accurately when the snowfall would start and stop.

Blame It On Rio

The IBM Research team is putting their algorithms to work on behalf of cities around the world. For instance, Rio de Janeiro, because of its climate and terrain, has recurring flooding and landslide problems in many hilly neighborhoods. The researchers used data describing the physics of the atmosphere to create a mathematical model of how storms are likely to unfold in Rio. With it, they can predict up to 40 hours ahead of time how much rain will fall in a particular location — with 90% accuracy.

In recent months, the Deep Thunder team, lead by Lloyd Treinish, has developed new techniques for ingesting many more measurements from weather sensors. The team is also extending its technology to new applications, including agriculture and wind farming.

For detailed and super-accurate weather information to have maximum impact, it has to be accessible by a large number of people. That’s why IBM has created iPad and cloud applications that deliver the power of Deep Thunder to people’s hands wherever they may be. Hopefully, by the time Rio hosts the summer Olympics in 2016, practically everybody who attends will be able to get their hands on Deep Thunder data so they know exactly what to expect when they venture out to the various game venues.

Note: This post originally appeared on IBM's Smarter Planet blog. IBM provided the storage hardware and software for the KMA project. 

Image courtesy of Shutterstock.

Yet for Africa to fulfill its potential and emerge among the world's economic tigers, social and business leaders agree that much more innovation must happen there. The continent's cities, universities, entrepreneurs and commercial R&D organizations can become engines of innovation producing new products and services that are tailored for the African experience. And, in order to make this transition, African institutions and businesses–plus multinational corporations –must work together to create innovation ecosystems that foster this kind of creativity.

At IBM's Smarter Planet Leadership Forum today in Nairobi, Kenya, CEO Ginni Rometty said IBM hopes to work collaboratively with the people and institutions in Africa: "We want to be seen as a citizen of the countries, essential to the government, companies and people." Rometty said IBM's decision to locate an IBM Research laboratory on the continent–beginning with an office in Nairobi–sends the strong signal about the company's commitment to Africa.

The message is clear: Rometty wants IBM to play an active role in building innovation ecosystems in Africa.

Taking Silicon Valley On The Road

California's Silicon Valley is the prototype innovation ecosystem. It benefitted from the combination of good universities, entrepreneurial companies, government incentives and robust supplies of venture capital. Many of other places have tried to copy Silicon Valley's formula—some quite successfully, among them Bangalore, India, and Singapore.

Kenya is among the countries in Africa that have the potential of creating a vibrant innovation ecosystem. Students and entrepreneurs dream of tapping science and technology to solve social and business problems. Universities aim to expand their research and teaching programs in science, math and technology. Business leaders are creating startup incubators to encourage entrepreneurship—places like iHub, FabLab Nairobi and NaiLab.

The government is playing a vital role, too, by making bold moves aimed at establishing Kenya as an information technology hub for East Africa. The government recently broke ground for Konzo Techno City, a new municipality being built from scratch south of Nairobi to bring research universities, corporations and government agencies together to support job creation, research collaboration and economic development. Another key move was the launch by the Kenya ICT Board of an incubation program for high-tech startups—including seed capital funding. "If we can build the skills and innovate, it will change the entire continent," said Bitange Ndemo, permanent secretary of Kenya's ministry of information and science.

Private Interests Play Key Role

But there's an important role for foreign companies and academic institutions to play, as well—as partners with African institutions in making progress. IBM Research's new research laboratory in Nairobi is the first basic scientific research lab to be established in Africa by a foreign multinational firm. We have allied with Catholic University of Eastern Africa to locate the lab on its Nairobi campus.The goal of the lab is to produce innovations within Africa and also bring in great ideas from IBM's other 11 research labs around the world. "We want to create technology solutions optimized for Africa that can be exported to the rest of the developing world," John Kelly, senior vice president and director of IBM Research said earlier this week.

While the first lab office is in Nairobi, IBM plans on expanding elsewhere around the continent and also performing collaborative research with a number of universities. Already, the company is engaging with the University of Nairobi and Strathmore University in collaborative programs where scientists from IBM will work with university faculty members on projects of mutual interest.

In another sign that Kenya is beginning to offer an attractive academic environment, Columbia University, one of the leading academic institutions in the United States, has set up Columbia Global Centre/Africa as a venue for research aimed at helping African nations reach their UN Millennium Development Goals.

We believe that foreign firms and institutions won't succeed if they try to build islands of expertise. They must work with local universities on collaborative research and to improve the quality of degree programs. Sure, if this happens it will mean that IBM Research will have to compete vigorously to recruit and retain the most skilled and ambitious young people. But so be it. "This is a long term investment," said Kelly. "We're here to help build the skills and, hopefully, we'll get our fair share of the most talented graduates."

Article and video courtesy of A Smarter Planet. Lead image courtesy of Shutterstock.

Across the continent, there’s a tremendous potential for using data analytics in powerful new ways in a wide range of industries and domains, from telecommunications and banking to transportation and healthcare.

IBM Chairman and CEO, Ginni Rometty at the IBM CIO Leadership Exchange in Johannesburg, South Africa, Feb. 5. The big data opportunity for Africa came into sharp focus this week when IBM CEO Ginni Rometty and key members of her executive team visited Africa to meet with clients and government leaders. “Going forward, data is going to be THE source of competitive advantage,” Rometty told a South African audience.

Already, African companies are harnessing big data to transform their businesses. Take Santam Ltd., the leading short-term insurance company in South Africa. The company is using predictive analysts to streamline the processing of claims and to spot potentially fraudulent claims.

When Anesh Govender reported for duty as Santam’s head of operations for finance, his boss told him that he wanted to do more with less. Govender quickly spotted data as his leverage point. “I was amazed how much data was available but how little of it was being used, ” he said. Govender was one of the presenters at IBM’s South African gathering.

He decided to completely overhaul the claims processing system. In the past, every claim went through the same steps of being reviewed manually by staff members. Today, they’re all fed into a predictive analytics software program that channels routine claims into a queue for quick action. The others go through deeper analysis that takes into account not just the current claim but a lot more information about the customer and their past claim activities. Computer algorithms search for patterns that suggests the claims might be fraudulent. One example: Fraudsters typically start with small false claims and, if they’re successful, submit larger ones.

Govender’s staff has tuned the algorithms so they identify the maximum amount of false claims without producing too many false positives—which require extra work by the claims processing staff. Today, they kick out only 1% of claims for deep fraud analysis, and about 30% of them are fraudulent.

The system has indeed made it possible for Santam to do more with less. While the volume of claims has risen 8% since the new system was put in place, the staff level has stayed level. Before, it took at least three days to settle legitimate claims. Now Santam clears them within an hour of receiving them.

Looking ahead, the most promising of the big data leapfrogging opportunities for Africa lie within the telecom industry. Mobile phones are now widespread, as are SMS based services such as mobile payments and money transfer. Next, observers expect rapid adoption of affordable smart phones capable of handling more sophisticated applications, such as social networking. It all adds up to a massive amount of data in the hands of the mobile carriers about what people are doing with their phones and where they are. “Businesses that find opportunities in data will do better,” says Michael Rhodin, senior vice president of IBM’s software solutions group. “These businesses will use the combination of data sourced from customer relationship management, operations and talent management.”

What will the carriers do with all that data? For starters, they can look for patterns in customers’ behavior that suggests they’re about to drop their service. So-called “churn” is expensive for carriers. But, alerted that particular customers are dissatisfied, carriers can approach them with offers that might entice them to stay on. Bharti Airtel, India’s largest mobile carrier, which now has a foothold on the African continent, already does this kind of analysis on its home turf.

As the mobile market continues to develop, the carriers will be in a strong position to offer their customers new applications and services. They’ll not only own the relationship with the customer, but they’ll see data patterns in customer behavior that helps them anticipate customer’s needs. Marketers have long dreamed of being able to address not just market segments but individual customers. Big data could finally turn this dream into a reality.

And it could happen first in Africa.

Article and Rometty image courtesy of A Smarter Planet. Lead image courtesy of Shutterstock.