The demand for text mining tools, services like Instapaper and Readability, and Web scraping have increased the importance of extracting article text from HTML pages.
Computer science student Toma Kova?i? wrote an overview of text extraction algorithms. He also a big list of resources for hackers working with text extraction, including research papers and articles, software and Web APIS.
Some of the techniques Kova?i? covers include:
- Boilerplate
- The Maximum Subsequence Optimization algorithm
- Text-to-Tag Ratio
- VIPS
- Readability
See also: our coverage of Extractiv, a text extraction and analysis service.
Image by Andrew Mason