Home Open-Source Search: Application Centric and a Way to Big Data

Open-Source Search: Application Centric and a Way to Big Data

Eric Schmidt took the stage today at TechCrunch Disrupt.

What he said seems striking, considering the pace of change we observe in our daily work. Schmidt said Moore’s Law gave us a guide for the speed to expect on processing information. He said what we did not expect is the amount of information we would produce.

What’s resulting in face of this big data are a number of alternatives to the Google way of search that we will learn a lot more about next week in Boston at Lucene Revolution, the first conference dedicated to open-source search.

What’s emerging is in some sense an application-centric approach to search that is built into services using open-source search technology such as Apache Solr and Lucene.

Developers have deeper needs for the applications they are developing. With the advent of big data, they often have more flexibility with open-source search technology as the data can be accessed via an API and indexed in Apache Solr or Lucene.

People from The Guardian will be attending Lucene Revolution. The Guardian is a modern newspaper – arguably one of the most innovative media companies of our time. It realizes it’s role is not to broadcast the news but be a part of the community as leaders in filtering information and providing applications and services.

RedMonk’s James Governor attended a NoSQL event at the Guardian a few months ago. Apache Solr was a focal part of discussion.

The database is part of the challenge. Oracle RAC has worked fine over the years for the Guardian. But now The Guardian has linked data, social networks and tags galore. The database is not designed for that load.

What happens with API access, which drives for example, tag proliferation, which dramatically increases load on the database.

“Apache Solr is like a database, it works like one for us.

Fields can be multi-value. one piece of content with five tags can be stored in one field. Most important is that SOLR offers the ability to facet the content. apply it *like* a tag…

For example: – an editor’s star rating. we can facet on that for free, and just jump to all the three star albums. facets can be combined much more quickly than a relational database.

With Solr we can perform complex queries, filter by facets.”

Open-source search will grow in demand as more applications depend on search technology for providing deeper context and meaning for the services they provide.

This means curating analytics and new ways to pull in, organize and index information.

We will be at the conference on Oct. 7-8. More than 250 people are expected for this new category of search conference. We plan to do some live streaming from the event. We’ll have more on that later this week.

Disclaimer: Lucid Imagination is paying for airfare, hotel and expenses for Alex Williams to attend Lucene Revolution.

About ReadWrite’s Editorial Process

The ReadWrite Editorial policy involves closely monitoring the tech industry for major developments, new product launches, AI breakthroughs, video game releases and other newsworthy events. Editors assign relevant stories to staff writers or freelance contributors with expertise in each particular topic area. Before publication, articles go through a rigorous round of editing for accuracy, clarity, and to ensure adherence to ReadWrite's style guidelines.

Get the biggest tech headlines of the day delivered to your inbox

    By signing up, you agree to our Terms and Privacy Policy. Unsubscribe anytime.

    Tech News

    Explore the latest in tech with our Tech News. We cut through the noise for concise, relevant updates, keeping you informed about the rapidly evolving tech landscape with curated content that separates signal from noise.

    In-Depth Tech Stories

    Explore tech impact in In-Depth Stories. Narrative data journalism offers comprehensive analyses, revealing stories behind data. Understand industry trends for a deeper perspective on tech's intricate relationships with society.

    Expert Reviews

    Empower decisions with Expert Reviews, merging industry expertise and insightful analysis. Delve into tech intricacies, get the best deals, and stay ahead with our trustworthy guide to navigating the ever-changing tech market.