Announcing Blogscape and SEOmoz Labs

Announcing Blogscape and SEOmoz Labs – Moz

Moz logo

Menu open

Menu close

Products

Moz Pro

Moz Pro Home

Moz Local

Moz Local Home

STAT

Mozscape API

Free SEO Tools

Competitive Research

Link Explorer

Keyword Explorer

Domain Analysis

MozBar

More Free SEO Tools

Learn SEO

Beginner’s Guide to SEO

SEO Learning Center

Moz Academy

SEO Q&A

Webinars, Whitepapers, & Guides

Blog

Why Moz

Agency Solutions

Enterprise Solutions

Small Business Solutions

Case Studies

The Moz Story

New Releases

Log out

Products

Moz Pro

Your All-In-One Suite of SEO Tools

The essential SEO toolset: keyword research, link building, site audits, page optimization, rank tracking, reporting, and more.

Learn more

Try Moz Pro free

Moz Local

Complete Local SEO Management

Raise your local SEO visibility with easy directory distribution, review management, listing updates, and more.

Learn more

Check my presence

STAT

Enterprise Rank Tracking

SERP tracking and analytics for SEO experts, STAT helps you stay competitive and agile with fresh insights.

Learn more

Book a demo

Mozscape API

The Power of Moz Data via API

Power your SEO with the proven, most accurate link metrics in the industry, powered by our index of trillions of links.

Learn more

Get connected

Compare SEO Products

Free SEO Tools

Competitive Research

Competitive Intelligence to Fuel Your SEO Strategy

Gain intel on your top SERP competitors, keyword gaps, and content opportunities.

Find competitors

Link Explorer

Powerful Backlink Data for SEO

Explore our index of over 40 trillion links to find backlinks, anchor text, Domain Authority, spam score, and more.

Get link data

Keyword Explorer

The One Keyword Research Tool for SEO Success

Discover the best traffic-driving keywords for your site from our index of over 500 million real keywords.

Search keywords

Domain Analysis

Free Domain SEO Analysis Tool

Get top competitive SEO metrics like Domain Authority, top pages, ranking keywords, and more.

Analyze domain

MozBar

Free, Instant SEO Metrics As You Surf

Using Google Chrome, see top SEO metrics instantly for any website or search result as you browse the web.

Try MozBar

More Free SEO Tools

Learn SEO

Beginner’s Guide to SEO
The #1 most popular introduction to SEO, trusted by millions.
Read the Beginner’s Guide

How-To Guides
Step-by-step guides to search success from the authority on SEO.
See All SEO Guides

SEO Learning Center
Broaden your knowledge with SEO resources for all skill levels.
Visit the Learning Center

Moz Academy
Upskill and get certified with on-demand courses & certifications.
Explore the Catalog

On-Demand Webinars
Learn modern SEO best practices from industry experts.
View All Webinars

SEO Q&A
Insights & discussions from an SEO community of 500,000+.
Find SEO Answers

August 7-9, 2023
Lock in Super Early Bird savings for MozCon

Snag tickets

Blog

Why Moz

Small Business Solutions
Uncover insights to make smarter marketing decisions in less time.
Grow Your Business

The Moz Story
Moz was the first & remains the most trusted SEO company.
Read Our Story

Agency Solutions
Earn & keep valuable clients with unparalleled data & insights.
Drive Client Success

Case Studies
Explore how Moz drives ROI with a proven track record of success.
See What’s Possible

Enterprise Solutions
Gain a competitive edge in the ever-changing world of search.
Scale Your SEO

New Releases
Get the scoop on the latest and greatest from Moz.
See What’s New

New Feature: Moz Pro
Surface actionable competitive intel

Learn More

Moz Pro

Moz Local

Moz Local Dashboard

Mozscape API

Mozscape API Dashboard

Moz Academy

Avatar

Moz Home

Notifications

Account & Billing

Manage Users

Community Profile

My Q&A

My Videos

Log Out

By: ChasW
March 2, 2009

Announcing Blogscape and SEOmoz Labs

Moz News

The author’s views are entirely his or her own (excluding the unlikely event of hypnosis) and may not always reflect the views of Moz.

Introduction

Hi, I’m Chas, a developer at SEOmoz. I’ve been working here since last summer, and though I’ve written many lines of code, this is my very first blog post. Now, I’m happy to have the chance to reveal the super-secret project I’ve been working on: Blogscape!

Blogscape

What are the hottest topics on the web right now? How much does your web presence change day-to-day, or in response to an advertising campaign? How many links did your site receive from the blogosphere this week?

Blogscape is a data source built to answer these questions and more. It’s an ‘Information Feed Aggregator’ and has been monitoring about 10 million feeds since December 2007. These feeds come from any website which offers syndication, but they are mostly from Blogs and News sites.

While Linkscape crawls the web-at-large, Blogscape is focused on the ‘fast-moving-web’. It stores and makes searchable the full content of syndication feeds (including link data!), and the newest data is made available several times every day.

Data from Blogscape will appear in various forms in upcoming SEOmoz products, but we’ve decided to release a version of our internal testing tool for it in…

SEOmoz Labs!

SEOmoz Labs is a place where our more adventurous PRO users can check out the ‘bleeding edge’ of SEOmoz technology and product design. From now on, Labs will be our showcase for new ideas and features, ranging from simple proof-of-concept tools to early releases of upcoming products. We’re hoping Labs will be a fun and exciting ‘sneak peek’ for our users – and a source of early feedback for our product development team.

There are some caveats associated with Labs – because these projects are works-in-progress, data from them could be inaccurate, and the projects themselves could become unavailable temporarily or permanently at any time. Beyond this, we are explicitly not offering user support for labs projects, so as to not take any time away from our ‘official projects’.

However, that doesn’t mean we don’t want to hear from you! In the upper right hand corner of each labs project page, there is a bright green button labeled “send feedback” (which is a link to labs@seomoz.org). Even though we won’t always be able to respond, we’d love to receive your comments, compliments, complaints, and bug reports. Feedback will be incorporated into our product design process, and labs projects with a significant amount of positive responses may find their way into formal projects sooner than others.

If you’re PRO and want to press on immediately, click here to check out SEOmoz Labs now! Otherwise, read on below for more details about Blogscape.

Queries and Graphs

The main Labs page for Blogscape has a box for queries, and a graph below to show how many posts match each query over time (the last 30 days, by default).

For example, the query
“Sean Penn”, “Brad Pitt”, “Richard Jenkins”, “Frank Langella”, “Mickey Rourke”
displays the number of mentions of each Academy Award Nominee for ‘Performance by an actor in a leading role’. The (somewhat cramped) snapshot below shows the results for this query (taken on Feb 25th, 2009, you can view the live version by clicking here).

So, this graph shows the number of posts which mention each actor’s name for every day in February. The spike in mentions for all queries on Feb 23rd corresponds with the actual date of the Academy Awards. As I’m sure you know, Sean Penn took the Oscar for this one, and this is clearly reflected in the fact that he received twice as many hits as any other query on that day.

Viewing Posts

Blogscape stores the snippets of text found in each Feed – you can click any data point to view this information. For example, if you’d clicked the on the line for Sean Penn on Feb 23rd, you’d see something like this:

This view shows snippets from each post satisfying that query on that day. Posts are ordered by Blogrank, an internally calculated ranking metric. (Any feed can ‘vote’ for any other feed by linking to the website the feed comes from. Feeds with more votes have higher Blogrank.)

Each post has the following information:
•    The original title of the post (clicking here takes you to the actual post)
•    A snippet of the description of the post
•    The title of the feed the post came from (clicking here takes you to the main page of the feed’s source)
•    The feed’s Blogrank
•    The URL for the feed itself
Advanced Queries

Beyond single terms or phrases in quotes, there are advanced query operators available. For example, you could search for posts containing the word ‘oscar’ or ‘oscars’ with the query
oscar | oscars (open this query)
There are also query operators for finding posts which link to specific URLs, root domains, or subdomains. For example, you could search for posts which link to any URL at the root domain ‘oscar.com’ with the query
rd:oscar.com (open this query)
A list of all available query operators can be found at the Blogscape help page.

Finally, each graph has option of being weighted by Blogrank (see checkbox on the right of the labs page). This makes the graph more of a measure of the ‘popularity’ of a query for any given day, instead of the raw number of matches for it. (Feeds with high Blogrank have many incoming links from other feeds, and tend to come from sources which are viewed by lots of people.)

Data Duplication

You may notice a message at the bottom of the ‘Posts’ view stating that “Posts very similar to these have been filtered from this list.” We’ve worked hard to battle data duplication in Blogscape by carefully canonicalizing feeds (many sites have several URLs for the same data) and posts within a feed. Nonetheless, there are situations where duplicate data is almost impossible to eliminate in advance (for example, some large sites have many feeds with content that occasionally overlaps).

To battle this problem, Blogscape does additional filtering of posts at query time. This filtering ensures that you see only the most relevant version of a post that occurs in Blogscape’s data stores multiple times. For this reason, some queries will have higher post counts on the frequency graphs than when viewing the Posts themselves. If you really want to view every post Blogscape has, you can click on the link at the bottom of the page to turn this feature off.

Data Quality

As I mentioned before, Blogscape has been monitoring a sizable portion of the Blogosphere for over a year. Nonetheless, we are striving to improve the quality of data within Blogscape, and we’ve very excited about two major upcoming improvements:
1. Monitoring of more high-quality feeds

We’ve added Feed Auto-Discovery logic to our processing of Linkscape crawl data, and will be using the results to make sure Blogscape always monitors the most important blogs from across the web.

2. Crawling of source pages

Based on our research, about half of syndication feeds don’t publish the entire content of their posts in their feeds – instead, they publish a truncated section of their content (or occasionally a hand-written summary of it). Most sites that do this also strip HTML from their feeds.
It’s important for data quality to ensure that queries for a term return all posts mentioning that term, and it’s important for SEO that all link information is present. For these reasons, we’re adding functionality to Blogscape that will follow links from syndication feeds, and store the actual source content for future search. (Of course, the upcoming crawler will politely ignore sites which block it using Robots.txt – details on this will be released when the crawler goes live.)

Movers and Shakers

Finally, an interesting use of the mountains of data stored by Blogscape is the search for hot trends, or ‘Movers and Shakers’. You can see the results of this process for several categories by clicking on the ‘Movers and Shakers’ links in the upper right hand corner of the Blogscape Labs page.

They tend to be most interesting (and stable) in weekly increments – you can view the top ‘mover and shaker’ phrases for this week here. On the day of writing this post, the top phrase is “Safari 4 beta,” which rose 26,632.4% this week (percent change over rank-weighted graphs). Right behind it is “Gary Locke,” which rose 25,101.7% over last week. On the labs page for this feature, you can click through and view the graph for each individual ‘mover and shaker’.

Conclusion

We’re excited to launch this feature, and even more excited about the data quality improvements we’ll be making on it in the next few months. If you’re PRO, check it out, and send your comments our way!

Snag your MozCon 2022 video bundle for even more SEO insights.

Buy the video bundle!

Why Legal Content Writing Is Important and How to Do It Right

The Long, Dark Teatime of Rand’s Decisions

Every Site is Linkbait & Linkerati Worthy

The Three Biggest Problems with PPC Ads

Facebook Contests, Sweepstakes and US Law: What You Need To Know

So You Want To Appear in Google’s Local Travel Guide?

类似文章