Home The RSS Connection: New Search, Big Data and the Web App Movement

The RSS Connection: New Search, Big Data and the Web App Movement

One thing the recession has done is fuel innovation by imposing financial constraints. The constraints have led to some dramatic market changes, in particular related to the rise of Lucene/Solr, the open-source search technology.

As a result, it feels like we are the beginning of something. It’s like we are witnessing the emergence of a new search. I tweeted that today about Lucene Revolution, the conference I am attending here in Boston. It is reminiscent of 2003, when blogs gave rise to RSS, a format designed to help us make sense of the online content that had started to flow like never before.

As RSS helped us make sense of information in 2003 so is Lucene/Solr helping make sense of data today. Big data is here to stay. It is increasingly fueling Web apps and is leading to new innovations in search, primarily in the open-source market. You see it here at Lucene Revolution. Twitter, LinkedIn, Salesforce.com and a host of others are discussing how the Web apps they have developed need Hadoop and Lucene/Solr to function properly for the apps to do what they want them to do.

RSS in many ways served as the precursor to a new breed of syndication technologies and the modern Web app, which gave rise to the activity stream. It’s a river of news, a term coined by Dave Winer, one of the Web’s pioneering developers . Twitter came out of this wave, be it indirectly, through the creations by Evan Williams and his team. Williams started Pyra Labs, which gave rise to Blogger. After selling to Google, Williams became captivated by the next incarnation of RSS. That was podcasting, which never really took off in a big way but perhaps more importantly served as a catalyst for a new wave of innovation in media delivery. Odeo flopped but Twitter rose from its ashes.

Twitter represented a new way to get news. It harnessed APIs like no other service had before. Developers embraced it. It marked a new wave, the app movement, which in turn has fueled the fast rise of social technologies. Text, pictures, videos are at the heart of these apps. The more we create, the more reason we give to developers to build new apps.

In the midst of this, the financial markets collapsed and with it a spiraling economy. Suddenly, it made even more sense to develop your own apps.

And here we are today. Budgets are still thin but the pipes are fat with content. So much so that we need new tools to make sense of it. What is coming from that is the start of something pretty significant. It’s the new search and it is helping us make sense of things.

Sometimes it feels like 2003.

About ReadWrite’s Editorial Process

The ReadWrite Editorial policy involves closely monitoring the tech industry for major developments, new product launches, AI breakthroughs, video game releases and other newsworthy events. Editors assign relevant stories to staff writers or freelance contributors with expertise in each particular topic area. Before publication, articles go through a rigorous round of editing for accuracy, clarity, and to ensure adherence to ReadWrite's style guidelines.

Get the biggest tech headlines of the day delivered to your inbox

    By signing up, you agree to our Terms and Privacy Policy. Unsubscribe anytime.

    Tech News

    Explore the latest in tech with our Tech News. We cut through the noise for concise, relevant updates, keeping you informed about the rapidly evolving tech landscape with curated content that separates signal from noise.

    In-Depth Tech Stories

    Explore tech impact in In-Depth Stories. Narrative data journalism offers comprehensive analyses, revealing stories behind data. Understand industry trends for a deeper perspective on tech's intricate relationships with society.

    Expert Reviews

    Empower decisions with Expert Reviews, merging industry expertise and insightful analysis. Delve into tech intricacies, get the best deals, and stay ahead with our trustworthy guide to navigating the ever-changing tech market.